-- Logs begin at Thu 2022-07-14 09:32:25 UTC, end at Thu 2022-07-14 10:32:02 UTC. -- Jul 14 09:59:39.924447 dv-cinder-849800-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 14 09:59:39.931874 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 14 09:59:39 2022] *** Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: nodename: dv-cinder-849800-1-iscsi Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: machine: x86_64 Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: clock source: unix Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: pcre jit disabled Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: detected number of CPU cores: 4 Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: current working directory: / Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: detected binary path: /usr/bin/uwsgi-core Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: your processes number limit is 47680 Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: your memory page size is 4096 bytes Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: detected max file descriptor number: 2048 Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: lock engine: pthread robust mutexes Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: thunder lock: enabled Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 14 09:59:39.937089 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 14 09:59:39.978936 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: Python main interpreter initialized at 0x5626528139c0 Jul 14 09:59:39.978936 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: python threads support enabled Jul 14 09:59:39.978936 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: your server socket listen backlog is limited to 100 connections Jul 14 09:59:39.978936 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: your mercy for graceful operations on workers is 90 seconds Jul 14 09:59:39.979320 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: mapped 403077 bytes (393 KB) for 2 cores Jul 14 09:59:39.979394 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: *** Operational MODE: preforking *** Jul 14 09:59:39.979624 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: *** uWSGI is running in multiple interpreter mode *** Jul 14 09:59:39.979624 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: spawned uWSGI master process (pid: 80316) Jul 14 09:59:39.979815 dv-cinder-849800-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 14 09:59:39.985121 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: spawned uWSGI worker 1 (pid: 80320, cores: 1) Jul 14 09:59:39.985121 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: spawned uWSGI worker 2 (pid: 80321, cores: 1) Jul 14 09:59:39.985121 dv-cinder-849800-1-iscsi devstack@keystone.service[80316]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 14 09:59:45.367725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 09:59:45.367725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: from cryptography.utils import int_from_bytes Jul 14 09:59:45.368492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 09:59:45.368492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: from cryptography.utils import int_from_bytes Jul 14 09:59:45.369707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 09:59:45.369707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: from cryptography.utils import int_from_bytes Jul 14 09:59:45.370161 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 14 09:59:45.370161 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: from cryptography.utils import int_from_bytes Jul 14 09:59:45.416321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 09:59:45.416509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 09:59:45.416748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] command line args: [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 09:59:45.417524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 09:59:45.417699 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ================================================================================ {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 09:59:45.417948 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] admin_token = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.418190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] config_dir = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.418372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.418549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] config_source = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.418723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] debug = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.418840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 14 09:59:45.418929 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.419236 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 14 09:59:45.419236 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] command line args: [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 14 09:59:45.419425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.419425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.419616 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 14 09:59:45.419717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ================================================================================ {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 14 09:59:45.419816 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.419916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] admin_token = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.420022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.420128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] list_limit = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.420227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] config_dir = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.420324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] log_config_append = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.420416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.420520 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.420621 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] config_source = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.420724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] log_dir = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.420824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] debug = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.420954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] log_file = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.421056 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.421183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.421279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.421399 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.421479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.421594 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.421668 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.421781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.421866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.421945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.422059 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] list_limit = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.422178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.422259 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] log_config_append = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.422380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.422464 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.422536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.422655 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] log_dir = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.422655 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] log_file = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.422836 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.422919 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.422992 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.423112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.423188 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.423305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.423386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.423552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.423637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.423768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.423849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.423921 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.424034 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.424119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.424360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.424521 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.424642 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.424730 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] publish_errors = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.424808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.424957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.425088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.425174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.425312 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.425396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.425550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.425632 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.425782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.425907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.426011 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] standard_threads = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.426099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.426236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.426475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.426551 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.61/identity {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.426643 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.426715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] publish_errors = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.426806 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] use_journal = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.426941 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.427017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] use_json = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.427154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] use_stderr = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.427233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.427304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] use_syslog = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.427373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.427507 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.427596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.427756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.427839 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.427988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.428071 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] standard_threads = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.428242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.428329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.428475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.428557 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.428657 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.428732 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.428838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] use_journal = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.428943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.429042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] use_json = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.429187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.429263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] use_stderr = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.429385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] use_syslog = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.429460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.429620 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 14 09:59:45.429705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.429931 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.430019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.430119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.430202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.430349 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.430426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.430590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.430667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.430822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.430904 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.431063 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.431141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.431303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.431386 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.431536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.431622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.431785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.431864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.432032 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.432109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.432296 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.432373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.432519 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.432593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.432767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.432846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.433029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.433109 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.433282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.433367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.433521 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.433599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.433748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.433833 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.433975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.434053 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.434204 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.434280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.434435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.434517 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.434679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.434762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.434922 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.434999 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.435149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.435227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.435375 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.435453 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.435616 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.435692 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.435856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.435940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.436089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.436165 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.436331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.436407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.436557 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.436646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.436788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.436865 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.437043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.437123 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.437272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.437361 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.437498 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.437577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.437724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.437800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.437949 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.438105 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.438180 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.438256 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.438415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.438492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.438642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.438733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.438874 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.438958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.439100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.439177 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.439335 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.439488 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.439569 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.439640 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.439783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.439860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.440009 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.440096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.440243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.440319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.440476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.440554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.440711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.440807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.440956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.441029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.441192 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.441265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.441414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.441534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.441720 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.441812 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.441929 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.442004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.442119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.442203 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.442306 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.442384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.442558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.442637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.442789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.442883 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.443017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.443097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.443249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.443332 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.443479 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.443560 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.443728 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.443816 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.443960 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.444080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.444197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.444274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.444431 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.444511 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.444660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.444742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.444891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.445025 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.445143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.445232 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.445388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.445473 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.445627 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.445713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.445894 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.445974 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.446134 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.446218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.446380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.446456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.446615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.446694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.446847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.446924 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.447080 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.447163 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.447311 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.447390 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.447568 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.447648 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.447804 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.447906 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.448065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.448145 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.448299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.448377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.448537 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.448615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.448773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.448852 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.449027 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.449108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.449262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.449339 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.449485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.449561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.449704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.449782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.449930 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.450008 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.450161 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.450248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.450386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.450465 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.450622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.450699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.450851 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.450927 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.451092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.451171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.451323 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.451401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.451573 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.451652 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.451842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.451927 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.452053 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.452130 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.452275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] auth.external = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.452352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.452500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.452580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.452740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.452817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.453000 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.453095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.453242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] auth.password = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.453320 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.453468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] auth.token = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.453546 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.453703 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.453789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.453934 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.454022 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.454178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.454262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.454405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.454492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] auth.external = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.454638 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.454724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.454880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.454964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.455107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.455185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.455338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] credential.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.455425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] auth.password = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.455579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.455658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] auth.token = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.455821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.455906 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.456057 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.456143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.456287 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.456364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.456525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.456604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.456755 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.456850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.457033 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.457114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.457270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.457347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.457499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.457587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] credential.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.457745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.457829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.457995 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.458073 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.458247 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.458337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.458486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.458564 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.458733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.458811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.458970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.459057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.459199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.459276 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.459432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.459518 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.459672 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.459759 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.459911 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.459989 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.460136 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.460215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.460376 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.460463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.460608 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.460687 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.460855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.462318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.462405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.462493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.462493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.462627 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.462698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.462698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.462831 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.462913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.462987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.462987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.462987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.463231 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.463304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.463383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.464044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.464129 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.464294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.464374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.464523 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.464679 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.464756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.464828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.465020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.465116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.465271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.465353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.465500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.465579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.465727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.465805 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.465953 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.466039 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.466193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.466273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.466436 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.466515 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.466677 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.466757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.466925 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.467003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.467165 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.467243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.467407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.467488 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.467648 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.467726 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.467882 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.467961 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.468109 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.468190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.468339 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.468416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.468563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.468680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.468794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.468878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.469051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.469132 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.469288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.469366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.469538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.469620 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.469767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.469847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.470010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.470089 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.470250 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.470330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.470481 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.470559 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.470722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.470800 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.470961 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.471041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.471190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.471267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.471424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.471501 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.471646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.471725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.471879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.471964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.472111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.472190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.472346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.472426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.472577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.472663 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.472810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.472889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.473081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.473171 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.473325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.473403 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.473554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.473909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.473992 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.474067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.474139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.474209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.474209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.474331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.474451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.474531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.474686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.474764 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.474915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.474992 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.475140 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.475220 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.475376 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.475454 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.475623 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.475701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.475847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.475932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.476087 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.476166 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.476318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.476395 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.476556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.476635 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.476787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.476866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.477035 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.477129 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.477268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.477347 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.477496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.477575 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.477726 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.477811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.477949 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.478026 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.478188 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.478266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.478427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.478513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.478666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.478745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.478905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.478983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.479191 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.479316 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.479921 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.480000 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.480155 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.480238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.480389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.480475 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.480814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.480902 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.480902 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.481100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.481177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.481334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.481414 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.481577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.481692 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.481820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.482148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.482304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.482384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.482550 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.482628 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.482782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.482860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.483021 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.483100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.483254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.483332 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.483486 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.483563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.483717 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.483797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.483952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.484029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.484187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.484264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.484415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.484495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.484650 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.484728 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.484884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.484992 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.485147 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.485227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.485389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.485466 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.485622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.485701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.485857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.485936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.486095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.486172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.486324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.486400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.486564 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.486645 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.486793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.486870 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.487025 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.487102 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.487255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] role.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.487336 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.487497 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] role.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.487576 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.487721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.487800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.487966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.488047 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.488206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.488284 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.488450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.488528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.488685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.488765 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.488940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.489022 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.489186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.489264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.489424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.489505 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.489660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] role.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.489739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.489885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.489962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] role.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.490127 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.490206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.490365 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.490443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.490607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.490698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.490848 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.490927 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.491093 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.491236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.491341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.491468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.491590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.491711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.491837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.491954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.492086 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.492197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.492320 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.492441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.492568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.492670 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.492816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.492904 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.493089 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.493170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.493338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.493417 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.493617 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.493729 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.493844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.493967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.494073 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.494219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.494320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.494468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.494570 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.494753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.494919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.495000 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.495158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.495246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.495394 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.495473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.495663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.495750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.495874 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.495955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.496112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.496191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.496355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.496434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.496586 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.496666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.496833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.496911 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.497099 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.497182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.497336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.497418 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.497577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.497656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.497818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.497897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.498052 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.498133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.498281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.498359 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.498514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.498630 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.498746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.498829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.498987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.499066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.499229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.499309 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.499454 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.499534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.499690 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.499769 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.499935 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.500014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.500181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.500262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.connection = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.500428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.500517 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.500667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.500747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.500897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.501007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.501161 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.501247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.501393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.501472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.501625 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.501704 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.501858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.501938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.502089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.502166 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.502329 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.502407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.502561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.502642 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.connection = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.502800 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.502889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.503031 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.503110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.503272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.503359 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.503504 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.503584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.503744 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.503823 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.503969 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.504056 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.504204 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.504285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.504435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.504513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.504605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80320) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 09:59:45.504693 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.504901 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.505189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.505426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.505673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.505911 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.506158 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.506389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.506629 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.506863 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 14 09:59:45.507036 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80321) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 14 09:59:45.534792 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.535016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.535240 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.536516 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.536754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.537016 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.544044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.545920 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.571418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.571642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.573457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.573693 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.712881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.714034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.763096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.763304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.763686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.763919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.827529 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.827743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.827877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.828029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.828167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.828308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.828441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.828573 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.828704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.828854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.829006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.829006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.853692 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.853956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.854111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.854248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.854381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.854514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.854646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.854786 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.854930 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.855061 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.855191 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.855191 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.855438 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.855584 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.866221 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.866477 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.878864 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.879293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.879479 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.879614 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.879614 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.879854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.879982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.880117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.880244 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.880371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.880371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.880609 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80321) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.880736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.880736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80320) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 14 09:59:45.897698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.897881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.898015 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 09:59:45.898143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 09:59:45.898282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.898411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.898538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 09:59:45.898664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 14 09:59:45.898903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.899068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.899216 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.899344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.899471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.899597 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.899723 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.899857 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.911188 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.911457 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.911721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.911998 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.912365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.912642 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.912911 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.913214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.920238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:45.921529 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:45.923545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:45.923815 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.924089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.924364 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.924640 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.924839 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:45.925007 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.925134 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.925264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.925392 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.925537 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.925658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.925741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.925922 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.926006 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.926183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.926273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.926447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.926720 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.926989 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.927260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.927525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.953361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.954889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.954976 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:45.955220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.956513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:45.956792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.960485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:45.960759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.961060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.962141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:45.962420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.962702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.969113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:45.969389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.970759 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:45.971043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.973874 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:45.974156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.974430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.974704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.974975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.975242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.975506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.975587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:45.975768 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.975851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.976080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.976363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.976640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.976912 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.977207 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.977476 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.995183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.995447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.997083 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:45.997361 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.000058 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.001995 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.003692 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.003981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.005753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.006047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.009154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.011804 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.013426 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.015457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.017024 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.017315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.017601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.017886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.018159 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.018438 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.019126 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.019426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.019727 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.020021 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.020313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.020594 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.037559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.037831 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.038094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.038349 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.039913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.040197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.040463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.040732 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.046271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.047963 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.048229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.048758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.050513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.050791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.051493 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.051768 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.052045 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.052335 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.052614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.054075 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.054356 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.054738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.055023 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.055311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.066413 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.066685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.066956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.067238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.067511 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.067792 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.068063 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.069255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.069536 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.069817 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.070100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.070383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.070672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.070947 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.087826 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.090904 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.091164 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.094247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.094504 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.096095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.097601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.097877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.098152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.098437 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.099224 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.101028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.101310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.101601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.105813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.107670 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.109262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.110898 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.111000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.111200 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.111478 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.111753 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.114169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.114463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.114749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.115024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.121563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.121842 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.122114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.124877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.125193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.125476 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.130259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80320) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.130534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.130814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.131081 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.131344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.131624 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.131892 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.132160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.132522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.132787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.133087 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.133349 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.133453 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80321) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 14 09:59:46.133619 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.133731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.133880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80320) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.134012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.134291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.134567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.135405 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.135683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.135958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.136330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.136614 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.136896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.137198 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.137470 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.137741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80321) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 14 09:59:46.172438 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5626528139c0 pid: 80320 (default app) Jul 14 09:59:46.175794 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5626528139c0 pid: 80321 (default app) Jul 14 09:59:46.175895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe895393-db1a-41bc-90ea-e3fe9b6ffa53 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:46.176088 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe895393-db1a-41bc-90ea-e3fe9b6ffa53 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:46.176362 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe895393-db1a-41bc-90ea-e3fe9b6ffa53 None None] PATH_INFO: `/v3` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:46.180544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1/1] 10.222.0.61 () {50 vars in 791 bytes} [Thu Jul 14 09:59:46 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:59:53.804513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2aeb5-5a5a-49fb-a0f3-49e61861d405 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:53.805087 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2aeb5-5a5a-49fb-a0f3-49e61861d405 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:53.805087 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e2aeb5-5a5a-49fb-a0f3-49e61861d405 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:53.809456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1/2] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 09:59:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:59:53.814581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18c67d-3633-4591-8224-c3e43a984b63 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:53.815008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18c67d-3633-4591-8224-c3e43a984b63 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:53.815135 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18c67d-3633-4591-8224-c3e43a984b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:53.883578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: INFO dbcounter [None req-cc18c67d-3633-4591-8224-c3e43a984b63 None None] Registered counter for database keystone Jul 14 09:59:53.905389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG oslo_db.sqlalchemy.engines [None req-cc18c67d-3633-4591-8224-c3e43a984b63 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80320) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 09:59:53.911636 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writer thread running {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 09:59:54.085121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG passlib.handlers.bcrypt [None req-cc18c67d-3633-4591-8224-c3e43a984b63 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80320) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 09:59:54.085454 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG passlib.handlers.bcrypt [None req-cc18c67d-3633-4591-8224-c3e43a984b63 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80320) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 09:59:54.139212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-cc18c67d-3633-4591-8224-c3e43a984b63 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:59:54.140825 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cc18c67d-3633-4591-8224-c3e43a984b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:59:54.261524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 2/3] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 09:59:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 448 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 14 09:59:54.425651 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2aeb5-5a5a-49fb-a0f3-49e61861d405 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:59:54.494913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: INFO dbcounter [None req-a9e2aeb5-5a5a-49fb-a0f3-49e61861d405 None None] Registered counter for database keystone Jul 14 09:59:54.514166 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG oslo_db.sqlalchemy.engines [None req-a9e2aeb5-5a5a-49fb-a0f3-49e61861d405 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80321) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 14 09:59:54.520390 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writer thread running {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 14 09:59:54.638304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e2aeb5-5a5a-49fb-a0f3-49e61861d405 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:59:54.642514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960066c-8c3d-4495-b186-66f0d48658b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:59:54.643198 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960066c-8c3d-4495-b186-66f0d48658b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:54.643475 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960066c-8c3d-4495-b186-66f0d48658b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:54.643748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960066c-8c3d-4495-b186-66f0d48658b8 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:54.651700 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960066c-8c3d-4495-b186-66f0d48658b8 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:59:54.704288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9960066c-8c3d-4495-b186-66f0d48658b8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 09:59:54.704570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960066c-8c3d-4495-b186-66f0d48658b8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:59:54.714086 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 2/4] 10.222.0.61 () {58 vars in 1324 bytes} [Thu Jul 14 09:59:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:59:54.719999 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18c67d-3633-4591-8224-c3e43a984b63 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:59:54.735240 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18c67d-3633-4591-8224-c3e43a984b63 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:59:54.740310 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b95eaf-311a-4a25-b8f2-5c6e7f61b710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:59:54.741460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b95eaf-311a-4a25-b8f2-5c6e7f61b710 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:54.741749 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b95eaf-311a-4a25-b8f2-5c6e7f61b710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:54.742031 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b95eaf-311a-4a25-b8f2-5c6e7f61b710 admin admin] PATH_INFO: `/v3/projects/4677fbdc96dd49fca8554ea63b0d5a7c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:54.743523 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b95eaf-311a-4a25-b8f2-5c6e7f61b710 admin admin] RBAC: Authorizing `identity:get_project(project_id=4677fbdc96dd49fca8554ea63b0d5a7c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:59:54.797121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: WARNING keystone.common.rbac_enforcer.enforcer [None req-76b95eaf-311a-4a25-b8f2-5c6e7f61b710 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 14 09:59:54.797471 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b95eaf-311a-4a25-b8f2-5c6e7f61b710 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:59:54.802700 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 3/5] 10.222.0.61 () {56 vars in 1207 bytes} [Thu Jul 14 09:59:54 2022] GET /identity/v3/projects/4677fbdc96dd49fca8554ea63b0d5a7c => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 09:59:56.809223 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1459d7-6d80-4b89-9777-72cf2e22dda4 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:56.809757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1459d7-6d80-4b89-9777-72cf2e22dda4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:56.809757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1459d7-6d80-4b89-9777-72cf2e22dda4 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:56.814021 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 3/6] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 09:59:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:59:56.819323 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e36a2-6f8a-4756-ac57-a61a52b434a7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:56.819706 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e36a2-6f8a-4756-ac57-a61a52b434a7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:56.819881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304e36a2-6f8a-4756-ac57-a61a52b434a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:56.838802 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-304e36a2-6f8a-4756-ac57-a61a52b434a7 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:59:56.840317 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-304e36a2-6f8a-4756-ac57-a61a52b434a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:59:56.849007 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 4/7] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 09:59:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 14 09:59:56.870068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1459d7-6d80-4b89-9777-72cf2e22dda4 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:59:56.882280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1459d7-6d80-4b89-9777-72cf2e22dda4 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:59:56.886351 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e06ec1-b0c6-4b56-a4fb-599c913d5b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:59:56.886972 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e06ec1-b0c6-4b56-a4fb-599c913d5b06 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:56.887249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e06ec1-b0c6-4b56-a4fb-599c913d5b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:56.887521 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e06ec1-b0c6-4b56-a4fb-599c913d5b06 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:56.895900 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e06ec1-b0c6-4b56-a4fb-599c913d5b06 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:59:56.897675 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e06ec1-b0c6-4b56-a4fb-599c913d5b06 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:59:56.906180 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 4/8] 10.222.0.61 () {58 vars in 1324 bytes} [Thu Jul 14 09:59:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 09:59:56.911147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e36a2-6f8a-4756-ac57-a61a52b434a7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:59:56.919962 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304e36a2-6f8a-4756-ac57-a61a52b434a7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:59:56.924492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e731ce-48f1-4368-ae3b-faa2e20c8d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:59:56.925391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e731ce-48f1-4368-ae3b-faa2e20c8d5e admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:56.925676 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e731ce-48f1-4368-ae3b-faa2e20c8d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:56.925949 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e731ce-48f1-4368-ae3b-faa2e20c8d5e admin admin] PATH_INFO: `/v3/users/e054eebdda2a494f804b604ef19ea604` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:56.927343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e731ce-48f1-4368-ae3b-faa2e20c8d5e admin admin] RBAC: Authorizing `identity:get_user(user_id=e054eebdda2a494f804b604ef19ea604)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:59:56.929185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e731ce-48f1-4368-ae3b-faa2e20c8d5e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:59:56.930759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 5/9] 10.222.0.61 () {56 vars in 1198 bytes} [Thu Jul 14 09:59:56 2022] GET /identity/v3/users/e054eebdda2a494f804b604ef19ea604 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:59:58.974255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa022e1-5f8c-44d1-a777-2ab3c3e72f69 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:58.974788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa022e1-5f8c-44d1-a777-2ab3c3e72f69 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:58.974788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa022e1-5f8c-44d1-a777-2ab3c3e72f69 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:58.976133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 5/10] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 09:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 09:59:58.981146 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6aa81-0696-44be-9e23-03e2e53f279e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:58.981538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6aa81-0696-44be-9e23-03e2e53f279e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:58.981830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d6aa81-0696-44be-9e23-03e2e53f279e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:58.998669 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-97d6aa81-0696-44be-9e23-03e2e53f279e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:59:59.000397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-97d6aa81-0696-44be-9e23-03e2e53f279e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:59:59.027182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23d4ac-e16a-4dca-ab52-f67777c10d34 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:59.027468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23d4ac-e16a-4dca-ab52-f67777c10d34 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:59.027743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c23d4ac-e16a-4dca-ab52-f67777c10d34 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:59.029121 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 6/11] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 09:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 09:59:59.034393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad8b6b-c07d-4ab5-9299-f10aa163af12 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:59.034676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad8b6b-c07d-4ab5-9299-f10aa163af12 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:59.034951 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad8b6b-c07d-4ab5-9299-f10aa163af12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:59.074266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG passlib.handlers.bcrypt [None req-4cad8b6b-c07d-4ab5-9299-f10aa163af12 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80321) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 14 09:59:59.074697 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG passlib.handlers.bcrypt [None req-4cad8b6b-c07d-4ab5-9299-f10aa163af12 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80321) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 14 09:59:59.082113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 6/12] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 09:59:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 09:59:59.104903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6aa81-0696-44be-9e23-03e2e53f279e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:59:59.121170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4cad8b6b-c07d-4ab5-9299-f10aa163af12 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 09:59:59.122765 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4cad8b6b-c07d-4ab5-9299-f10aa163af12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 09:59:59.128155 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:59:59.206813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d6aa81-0696-44be-9e23-03e2e53f279e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:59:59.221543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 7/13] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 09:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 09:59:59.240708 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098ffd4-2a27-4d67-9c33-fe1b05a58758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:59:59.241478 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098ffd4-2a27-4d67-9c33-fe1b05a58758 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:59.241770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098ffd4-2a27-4d67-9c33-fe1b05a58758 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:59.242051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4098ffd4-2a27-4d67-9c33-fe1b05a58758 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:59.242834 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad8b6b-c07d-4ab5-9299-f10aa163af12 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:59:59.254515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4098ffd4-2a27-4d67-9c33-fe1b05a58758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-4098ffd4-2a27-4d67-9c33-fe1b05a58758 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 09:59:59.266571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:59:59.267526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:59:59.268333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:59:59.268333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:59:59.268333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:59:59.268333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:59:59.268333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 09:59:59.268333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 09:59:59.268333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 09:59:59.268333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 09:59:59.268333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 09:59:59.268333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 09:59:59.268333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 09:59:59.268333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 09:59:59.268333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 7/14] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 09:59:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 09:59:59.273474 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098ffd4-2a27-4d67-9c33-fe1b05a58758 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:59:59.343181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad8b6b-c07d-4ab5-9299-f10aa163af12 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:59:59.345760 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:59:59.355121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:59:59.373452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4098ffd4-2a27-4d67-9c33-fe1b05a58758 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:59:59.381517 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eb6ed-a190-4d55-8da2-290ee694928b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:59:59.382418 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eb6ed-a190-4d55-8da2-290ee694928b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:59.382698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eb6ed-a190-4d55-8da2-290ee694928b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:59.382970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48eb6ed-a190-4d55-8da2-290ee694928b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:59.407618 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2119a-a46e-40f3-9299-387b71ab858f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:59:59.408279 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2119a-a46e-40f3-9299-387b71ab858f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:59.408561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2119a-a46e-40f3-9299-387b71ab858f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:59.408838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c2119a-a46e-40f3-9299-387b71ab858f None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:59.409621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2119a-a46e-40f3-9299-387b71ab858f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:59:59.409923 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2119a-a46e-40f3-9299-387b71ab858f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:59:59.411317 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c2119a-a46e-40f3-9299-387b71ab858f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:59:59.422753 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 8/15] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 09:59:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:59:59.428254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2119a-a46e-40f3-9299-387b71ab858f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:59:59.449310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48eb6ed-a190-4d55-8da2-290ee694928b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:59:59.450829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48eb6ed-a190-4d55-8da2-290ee694928b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:59:59.527911 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c2119a-a46e-40f3-9299-387b71ab858f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:59:59.548283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 8/16] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 09:59:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 09:59:59.553822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eb6ed-a190-4d55-8da2-290ee694928b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:59:59.556423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:59:59.562229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43735ad-4feb-440c-a2c6-a7606ed7609b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:59:59.563129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43735ad-4feb-440c-a2c6-a7606ed7609b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:59.563393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43735ad-4feb-440c-a2c6-a7606ed7609b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:59.563667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43735ad-4feb-440c-a2c6-a7606ed7609b None admin] PATH_INFO: `/v3/users/e054eebdda2a494f804b604ef19ea604` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:59.564892 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43735ad-4feb-440c-a2c6-a7606ed7609b None admin] RBAC: Authorizing `identity:get_user(user_id=e054eebdda2a494f804b604ef19ea604)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:59:59.566386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43735ad-4feb-440c-a2c6-a7606ed7609b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:59:59.568001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 9/17] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 09:59:59 2022] GET /identity/v3/users/e054eebdda2a494f804b604ef19ea604 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:59:59.574787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43735ad-4feb-440c-a2c6-a7606ed7609b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:59:59.587398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:59:59.656387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48eb6ed-a190-4d55-8da2-290ee694928b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:59:59.671484 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43735ad-4feb-440c-a2c6-a7606ed7609b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:59:59.690292 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ad800-5fac-453e-b000-2c12c5d53bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:59:59.691088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ad800-5fac-453e-b000-2c12c5d53bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:59.691370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ad800-5fac-453e-b000-2c12c5d53bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:59.691649 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27ad800-5fac-453e-b000-2c12c5d53bc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:59.705106 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c65a6-7f53-485e-8d64-2ccd064f1485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:59:59.705894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c65a6-7f53-485e-8d64-2ccd064f1485 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:59.706176 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c65a6-7f53-485e-8d64-2ccd064f1485 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:59.706448 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1c65a6-7f53-485e-8d64-2ccd064f1485 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:59.707633 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c65a6-7f53-485e-8d64-2ccd064f1485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:59:59.709110 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1c65a6-7f53-485e-8d64-2ccd064f1485 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:59:59.709433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ad800-5fac-453e-b000-2c12c5d53bc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:59:59.710996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 10/18] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 09:59:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:59:59.711218 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27ad800-5fac-453e-b000-2c12c5d53bc6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:59:59.716528 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c65a6-7f53-485e-8d64-2ccd064f1485 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:59:59.733204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-f27ad800-5fac-453e-b000-2c12c5d53bc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:59:59.733204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 09:59:59.733204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 09:59:59.733204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 09:59:59.733204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 09:59:59.733204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 09:59:59.733204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 09:59:59.733204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 09:59:59.733204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 09:59:59.733204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 09:59:59.733204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 09:59:59.733204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 09:59:59.733204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 09:59:59.733204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 09:59:59.733204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 09:59:59.734243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 09:59:59.735514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 09:59:59.735514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 09:59:59.735514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 09:59:59.735514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 09:59:59.735514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 9/19] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 09:59:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 09:59:59.740045 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ad800-5fac-453e-b000-2c12c5d53bc6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 09:59:59.796600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:59:59.799948 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 09:59:59.820664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1c65a6-7f53-485e-8d64-2ccd064f1485 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:59:59.841095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27ad800-5fac-453e-b000-2c12c5d53bc6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 09:59:59.854310 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440dfc6d-85ff-47b8-8b03-5903f078aa64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:59:59.854971 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440dfc6d-85ff-47b8-8b03-5903f078aa64 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:59.855255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440dfc6d-85ff-47b8-8b03-5903f078aa64 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:59.855528 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440dfc6d-85ff-47b8-8b03-5903f078aa64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:59.856361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440dfc6d-85ff-47b8-8b03-5903f078aa64 None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.domain.id=default, user.id=e054eebdda2a494f804b604ef19ea604) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:59:59.856669 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440dfc6d-85ff-47b8-8b03-5903f078aa64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:59:59.858679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440dfc6d-85ff-47b8-8b03-5903f078aa64 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:59:59.865681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 11/20] 10.222.0.61 () {56 vars in 1322 bytes} [Thu Jul 14 09:59:59 2022] GET /identity/v3/role_assignments?user.id=e054eebdda2a494f804b604ef19ea604&scope.domain.id=default&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 09:59:59.876072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9ee2e-7b86-487f-a73d-4928567345ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 09:59:59.876856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9ee2e-7b86-487f-a73d-4928567345ef None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 09:59:59.877233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9ee2e-7b86-487f-a73d-4928567345ef None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 09:59:59.877592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9ee2e-7b86-487f-a73d-4928567345ef None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 09:59:59.878413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9ee2e-7b86-487f-a73d-4928567345ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 09:59:59.878785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9ee2e-7b86-487f-a73d-4928567345ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 09:59:59.880293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e9ee2e-7b86-487f-a73d-4928567345ef None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 09:59:59.899863 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 10/21] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 09:59:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:02.313641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a35b1-f83a-4328-b148-3e98a69c2135 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:02.314598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a35b1-f83a-4328-b148-3e98a69c2135 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:02.315111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5a35b1-f83a-4328-b148-3e98a69c2135 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:02.316708 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 12/22] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:02.323055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc981a2a-b3fa-4289-ac05-4b7b09cad35b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:02.323853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc981a2a-b3fa-4289-ac05-4b7b09cad35b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:02.324324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc981a2a-b3fa-4289-ac05-4b7b09cad35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:02.350452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-fc981a2a-b3fa-4289-ac05-4b7b09cad35b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:02.352014 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fc981a2a-b3fa-4289-ac05-4b7b09cad35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:02.442048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 11/23] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:02.476651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a35b1-f83a-4328-b148-3e98a69c2135 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:02.598551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5a35b1-f83a-4328-b148-3e98a69c2135 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:02.642554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba257e6-b617-4c81-a3d2-8cc975ab8c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:02.643725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba257e6-b617-4c81-a3d2-8cc975ab8c01 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:02.644204 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba257e6-b617-4c81-a3d2-8cc975ab8c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:02.644672 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba257e6-b617-4c81-a3d2-8cc975ab8c01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:02.651024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ae32d-e8fa-4da6-a49e-596b4005cf43 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:02.651579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ae32d-e8fa-4da6-a49e-596b4005cf43 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:02.652050 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90ae32d-e8fa-4da6-a49e-596b4005cf43 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:02.654259 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 12/24] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:00:02.660859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba257e6-b617-4c81-a3d2-8cc975ab8c01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:02.662733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba257e6-b617-4c81-a3d2-8cc975ab8c01 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:02.666190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f5855-9e7a-4136-b766-826766ffd66f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:02.666686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f5855-9e7a-4136-b766-826766ffd66f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:02.667170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f5855-9e7a-4136-b766-826766ffd66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:02.680164 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:02.690901 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f90f5855-9e7a-4136-b766-826766ffd66f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:02.692109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f90f5855-9e7a-4136-b766-826766ffd66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:02.704720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-8ba257e6-b617-4c81-a3d2-8cc975ab8c01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:00:02.704720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:02.704720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:02.704720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:02.704720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:02.704720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:02.704720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:02.704720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:02.704720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:02.704720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:02.704720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:02.704720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:02.704720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:00:02.704720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:00:02.704720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:02.705605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:02.706427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:00:02.706427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:00:02.706427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:00:02.706427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:02.715996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 13/25] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:02.720489 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba257e6-b617-4c81-a3d2-8cc975ab8c01 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:02.756610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:02.789062 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 13/26] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:02.794732 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e910ac-c1ce-4e1b-9ce2-559187db8784 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:02.795048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e910ac-c1ce-4e1b-9ce2-559187db8784 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:02.795332 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e910ac-c1ce-4e1b-9ce2-559187db8784 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:02.800997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 14/27] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:02 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:02.802176 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febe8e9-e6eb-4e3d-9266-076b3b6f2e41 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:02.802684 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febe8e9-e6eb-4e3d-9266-076b3b6f2e41 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:02.802965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febe8e9-e6eb-4e3d-9266-076b3b6f2e41 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:02.807918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 15/28] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:02.809808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a7597-7dfd-48f4-a6c7-01903b7b0022 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:02.810096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a7597-7dfd-48f4-a6c7-01903b7b0022 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:02.810374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a7597-7dfd-48f4-a6c7-01903b7b0022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:02.833755 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c68a7597-7dfd-48f4-a6c7-01903b7b0022 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:02.835431 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c68a7597-7dfd-48f4-a6c7-01903b7b0022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:02.854057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba257e6-b617-4c81-a3d2-8cc975ab8c01 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:02.890537 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b7eca7-710f-422f-be7b-92531a02c699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:02.891546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7eca7-710f-422f-be7b-92531a02c699 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:02.892023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7eca7-710f-422f-be7b-92531a02c699 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:02.892470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b7eca7-710f-422f-be7b-92531a02c699 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:02.893468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7eca7-710f-422f-be7b-92531a02c699 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:02.893963 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7eca7-710f-422f-be7b-92531a02c699 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:02.895525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b7eca7-710f-422f-be7b-92531a02c699 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:02.912007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 16/29] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:02.916258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 14/30] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:02.918618 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559db180-76ed-4995-b0b2-d0ac615d8a0d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:02.919058 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559db180-76ed-4995-b0b2-d0ac615d8a0d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:02.919467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559db180-76ed-4995-b0b2-d0ac615d8a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:02.922012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a7597-7dfd-48f4-a6c7-01903b7b0022 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:02.937962 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-559db180-76ed-4995-b0b2-d0ac615d8a0d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:02.939376 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-559db180-76ed-4995-b0b2-d0ac615d8a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:02.992981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:03.012199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 15/31] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:03.013394 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559db180-76ed-4995-b0b2-d0ac615d8a0d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:03.052851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a7597-7dfd-48f4-a6c7-01903b7b0022 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:03.061176 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:03.091443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6a51d-102d-45ee-a1a6-997521957f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:03.092394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6a51d-102d-45ee-a1a6-997521957f7e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:03.093064 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6a51d-102d-45ee-a1a6-997521957f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:03.093445 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6a51d-102d-45ee-a1a6-997521957f7e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:03.106829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6a51d-102d-45ee-a1a6-997521957f7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-85d6a51d-102d-45ee-a1a6-997521957f7e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:03.116410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:03.117373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:03.118226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:03.118226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:03.118226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:03.118226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:03.118226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:03.118226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:03.118226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:03.118226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:03.118226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:03.118226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:03.118226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:03.118226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:03.118226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 17/32] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:00:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:03.123955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6a51d-102d-45ee-a1a6-997521957f7e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:03.125618 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559db180-76ed-4995-b0b2-d0ac615d8a0d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:03.161651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a99a89-9b05-4ac7-9cb3-b9643f2244b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:03.162667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a99a89-9b05-4ac7-9cb3-b9643f2244b5 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:03.163117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a99a89-9b05-4ac7-9cb3-b9643f2244b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:03.163586 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a99a89-9b05-4ac7-9cb3-b9643f2244b5 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:03.164536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a99a89-9b05-4ac7-9cb3-b9643f2244b5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:03.166183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a99a89-9b05-4ac7-9cb3-b9643f2244b5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:03.233593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6a51d-102d-45ee-a1a6-997521957f7e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:03.281440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b640560-497b-42c6-bb88-722883de3829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:03.282241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b640560-497b-42c6-bb88-722883de3829 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:03.282598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b640560-497b-42c6-bb88-722883de3829 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:03.282953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b640560-497b-42c6-bb88-722883de3829 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:03.283774 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b640560-497b-42c6-bb88-722883de3829 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:03.285279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b640560-497b-42c6-bb88-722883de3829 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:03.311536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 16/33] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Jul 14 10:00:03 2022] POST /identity/v3/projects => generated 301 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:00:03.313010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a99a89-9b05-4ac7-9cb3-b9643f2244b5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:03.360273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:03.423830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 18/34] 10.222.0.61 () {60 vars in 1131 bytes} [Thu Jul 14 10:00:03 2022] POST /identity/v3/roles => generated 219 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:00:03.429124 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b640560-497b-42c6-bb88-722883de3829 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:03.430760 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a99a89-9b05-4ac7-9cb3-b9643f2244b5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:03.471336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d972c0d4-b6eb-479d-afe6-2a75a2e33ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:03.472406 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972c0d4-b6eb-479d-afe6-2a75a2e33ea2 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:03.472896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972c0d4-b6eb-479d-afe6-2a75a2e33ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:03.473395 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d972c0d4-b6eb-479d-afe6-2a75a2e33ea2 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:03.474409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972c0d4-b6eb-479d-afe6-2a75a2e33ea2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:03.476374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d972c0d4-b6eb-479d-afe6-2a75a2e33ea2 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:03.483428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:03.504521 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 17/35] 10.222.0.61 () {60 vars in 1131 bytes} [Thu Jul 14 10:00:03 2022] POST /identity/v3/roles => generated 225 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:00:03.518067 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8b0e8-69b8-4b69-8730-955586c2786b None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:03.518680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8b0e8-69b8-4b69-8730-955586c2786b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:03.519185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c8b0e8-69b8-4b69-8730-955586c2786b None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:03.520871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 18/36] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:00:03.526515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227a4b9-5405-4cc5-bc29-91edae1c31e1 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:03.527705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227a4b9-5405-4cc5-bc29-91edae1c31e1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:03.528194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b227a4b9-5405-4cc5-bc29-91edae1c31e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:03.589009 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b640560-497b-42c6-bb88-722883de3829 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:03.630100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6715a1-0533-4b35-a837-b6c38c417332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:03.630741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6715a1-0533-4b35-a837-b6c38c417332 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:03.631032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6715a1-0533-4b35-a837-b6c38c417332 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:03.631326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6715a1-0533-4b35-a837-b6c38c417332 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:03.632126 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6715a1-0533-4b35-a837-b6c38c417332 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:03.632444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6715a1-0533-4b35-a837-b6c38c417332 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:03.634172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6715a1-0533-4b35-a837-b6c38c417332 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:03.650130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 19/37] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:03.656431 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6715a1-0533-4b35-a837-b6c38c417332 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:03.738560 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b227a4b9-5405-4cc5-bc29-91edae1c31e1 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:03.740098 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b227a4b9-5405-4cc5-bc29-91edae1c31e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:03.768702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:03.785823 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6715a1-0533-4b35-a837-b6c38c417332 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:03.831972 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd724ec0-989b-4991-a98b-5a38e9b28f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:03.832193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 19/38] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:03.833066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd724ec0-989b-4991-a98b-5a38e9b28f52 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:03.833353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd724ec0-989b-4991-a98b-5a38e9b28f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:03.833635 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd724ec0-989b-4991-a98b-5a38e9b28f52 None admin] PATH_INFO: `/v3/users/e054eebdda2a494f804b604ef19ea604` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:03.835366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd724ec0-989b-4991-a98b-5a38e9b28f52 None admin] RBAC: Authorizing `identity:get_user(user_id=e054eebdda2a494f804b604ef19ea604)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:03.836907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd724ec0-989b-4991-a98b-5a38e9b28f52 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:03.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 20/39] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:03 2022] GET /identity/v3/users/e054eebdda2a494f804b604ef19ea604 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:03.845037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227a4b9-5405-4cc5-bc29-91edae1c31e1 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:03.862015 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd724ec0-989b-4991-a98b-5a38e9b28f52 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:03.878281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:03.961267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b227a4b9-5405-4cc5-bc29-91edae1c31e1 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:03.977264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd724ec0-989b-4991-a98b-5a38e9b28f52 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:04.002244 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197f0d5-6e22-4bc4-a682-1757220d2345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:04.003345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197f0d5-6e22-4bc4-a682-1757220d2345 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:04.003761 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197f0d5-6e22-4bc4-a682-1757220d2345 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:04.004193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197f0d5-6e22-4bc4-a682-1757220d2345 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:04.005742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197f0d5-6e22-4bc4-a682-1757220d2345 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:04.007187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197f0d5-6e22-4bc4-a682-1757220d2345 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:04.009235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 20/40] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:00:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:04.013565 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956bb23-7362-40b7-a709-3fa98cb1345e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:04.014252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956bb23-7362-40b7-a709-3fa98cb1345e None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:04.014530 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197f0d5-6e22-4bc4-a682-1757220d2345 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:04.014626 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956bb23-7362-40b7-a709-3fa98cb1345e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:04.014816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956bb23-7362-40b7-a709-3fa98cb1345e None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:04.015587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956bb23-7362-40b7-a709-3fa98cb1345e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:04.016998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956bb23-7362-40b7-a709-3fa98cb1345e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:04.032602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 21/41] 10.222.0.61 () {60 vars in 1131 bytes} [Thu Jul 14 10:00:03 2022] POST /identity/v3/roles => generated 223 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:00:04.065527 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d436ec-7b2d-4b14-b796-f1f4ec75050a None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:04.065850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d436ec-7b2d-4b14-b796-f1f4ec75050a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:04.066097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d436ec-7b2d-4b14-b796-f1f4ec75050a None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:04.067519 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 22/42] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:00:04.073397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22363e74-08f4-483f-b093-424afc744271 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:04.073679 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22363e74-08f4-483f-b093-424afc744271 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:04.073963 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22363e74-08f4-483f-b093-424afc744271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:04.091622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-22363e74-08f4-483f-b093-424afc744271 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:04.093527 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-22363e74-08f4-483f-b093-424afc744271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:04.106820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:04.140791 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:04.147586 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197f0d5-6e22-4bc4-a682-1757220d2345 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:04.171173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 23/43] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:04.188543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee3a33-dd2f-47df-a696-e0e5c6f6177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:04.189469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee3a33-dd2f-47df-a696-e0e5c6f6177f None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:04.189741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee3a33-dd2f-47df-a696-e0e5c6f6177f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:04.190020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ee3a33-dd2f-47df-a696-e0e5c6f6177f None admin] PATH_INFO: `/v3/domains/default/users/e054eebdda2a494f804b604ef19ea604/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:04.192427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee3a33-dd2f-47df-a696-e0e5c6f6177f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e054eebdda2a494f804b604ef19ea604, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:04.193688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ee3a33-dd2f-47df-a696-e0e5c6f6177f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:04.194217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22363e74-08f4-483f-b093-424afc744271 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:04.212094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 21/44] 10.222.0.61 () {58 vars in 1361 bytes} [Thu Jul 14 10:00:04 2022] PUT /identity/v3/domains/default/users/e054eebdda2a494f804b604ef19ea604/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:00:04.312100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22363e74-08f4-483f-b093-424afc744271 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:04.347546 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955d3f3-4559-4018-b219-cee6ebe862b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:04.348356 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955d3f3-4559-4018-b219-cee6ebe862b6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:04.348636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955d3f3-4559-4018-b219-cee6ebe862b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:04.348907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955d3f3-4559-4018-b219-cee6ebe862b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:04.350144 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955d3f3-4559-4018-b219-cee6ebe862b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:04.351611 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b955d3f3-4559-4018-b219-cee6ebe862b6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:04.353292 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 24/45] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:00:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:04.359783 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee3a33-dd2f-47df-a696-e0e5c6f6177f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:04.460937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ee3a33-dd2f-47df-a696-e0e5c6f6177f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:04.494900 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3445150-f78a-465e-a17a-97b727ade3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:04.495499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3445150-f78a-465e-a17a-97b727ade3eb None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:04.495784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3445150-f78a-465e-a17a-97b727ade3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:04.496064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3445150-f78a-465e-a17a-97b727ade3eb None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:04.496793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3445150-f78a-465e-a17a-97b727ade3eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:04.498272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3445150-f78a-465e-a17a-97b727ade3eb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:04.522855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 22/46] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Jul 14 10:00:04 2022] POST /identity/v3/projects => generated 312 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:00:06.237336 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d134660-203a-4af9-b8e9-25a916f39f48 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:06.237856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d134660-203a-4af9-b8e9-25a916f39f48 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:06.237941 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d134660-203a-4af9-b8e9-25a916f39f48 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:06.239424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 25/47] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:06.245133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10824bd5-4f17-4e9b-bf59-f45c30cde574 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:06.245567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10824bd5-4f17-4e9b-bf59-f45c30cde574 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:06.245707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10824bd5-4f17-4e9b-bf59-f45c30cde574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:06.262206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-10824bd5-4f17-4e9b-bf59-f45c30cde574 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:06.263423 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-10824bd5-4f17-4e9b-bf59-f45c30cde574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:06.265404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:06.331417 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 23/48] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:06.355095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d134660-203a-4af9-b8e9-25a916f39f48 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:06.371018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:06.549864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad6a28-0a05-4614-8d0a-bae6e494206d None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:06.550074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad6a28-0a05-4614-8d0a-bae6e494206d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:06.550368 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad6a28-0a05-4614-8d0a-bae6e494206d None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:06.551754 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 24/49] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:06.557192 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f6bdf-89fd-4e49-994f-daf21a5d7512 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:06.557488 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f6bdf-89fd-4e49-994f-daf21a5d7512 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:06.557773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f6bdf-89fd-4e49-994f-daf21a5d7512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:06.560500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d134660-203a-4af9-b8e9-25a916f39f48 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:06.574114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7b4f6bdf-89fd-4e49-994f-daf21a5d7512 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:06.575301 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7b4f6bdf-89fd-4e49-994f-daf21a5d7512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:06.594212 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c22065-ea37-4c35-8b9e-97b9411ce6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:06.595008 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c22065-ea37-4c35-8b9e-97b9411ce6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:06.595297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c22065-ea37-4c35-8b9e-97b9411ce6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:06.595578 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c22065-ea37-4c35-8b9e-97b9411ce6c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:06.606211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c22065-ea37-4c35-8b9e-97b9411ce6c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-e0c22065-ea37-4c35-8b9e-97b9411ce6c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:06.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:06.614827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:06.615596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:06.615596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:06.615596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:06.615596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:06.615596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:06.615596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:06.615596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:06.615596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:06.615596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:06.615596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:06.615596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:06.615596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:06.615596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 26/50] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:00:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:00:06.621559 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c22065-ea37-4c35-8b9e-97b9411ce6c2 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:06.642086 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 25/51] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:06.664837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f6bdf-89fd-4e49-994f-daf21a5d7512 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:06.693846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:06.721536 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c22065-ea37-4c35-8b9e-97b9411ce6c2 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:06.732737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:06.755602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327eee3-8844-466a-957e-d8bc24dc2a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:06.756271 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327eee3-8844-466a-957e-d8bc24dc2a16 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:06.756572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327eee3-8844-466a-957e-d8bc24dc2a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:06.756837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7327eee3-8844-466a-957e-d8bc24dc2a16 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:06.757648 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327eee3-8844-466a-957e-d8bc24dc2a16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:06.757963 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327eee3-8844-466a-957e-d8bc24dc2a16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:06.759486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7327eee3-8844-466a-957e-d8bc24dc2a16 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:06.767068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f6bdf-89fd-4e49-994f-daf21a5d7512 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:06.768179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 27/52] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:06.773577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327eee3-8844-466a-957e-d8bc24dc2a16 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:06.803605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2f8d4-b9ef-4dac-ba91-ab6886de6fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:06.804331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2f8d4-b9ef-4dac-ba91-ab6886de6fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:06.804864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2f8d4-b9ef-4dac-ba91-ab6886de6fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:06.804864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e2f8d4-b9ef-4dac-ba91-ab6886de6fd5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:06.806128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e2f8d4-b9ef-4dac-ba91-ab6886de6fd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:06.807589 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e2f8d4-b9ef-4dac-ba91-ab6886de6fd5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:06.813758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 26/53] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:00:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:06.814950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2f8d4-b9ef-4dac-ba91-ab6886de6fd5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:06.896621 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7327eee3-8844-466a-957e-d8bc24dc2a16 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:06.926754 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e2f8d4-b9ef-4dac-ba91-ab6886de6fd5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:06.931213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32acce2-83a6-43a9-a677-72d625e70199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:06.932258 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32acce2-83a6-43a9-a677-72d625e70199 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:06.932684 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32acce2-83a6-43a9-a677-72d625e70199 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:06.933128 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32acce2-83a6-43a9-a677-72d625e70199 None admin] PATH_INFO: `/v3/users/e054eebdda2a494f804b604ef19ea604` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:06.934522 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32acce2-83a6-43a9-a677-72d625e70199 None admin] RBAC: Authorizing `identity:get_user(user_id=e054eebdda2a494f804b604ef19ea604)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:06.936108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32acce2-83a6-43a9-a677-72d625e70199 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:06.937774 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 28/54] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:06 2022] GET /identity/v3/users/e054eebdda2a494f804b604ef19ea604 => generated 247 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:06.943034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32acce2-83a6-43a9-a677-72d625e70199 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:06.947023 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:06.965581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef462604-ca83-47eb-aff4-37f27be5ae03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:06.966789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef462604-ca83-47eb-aff4-37f27be5ae03 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:06.967326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef462604-ca83-47eb-aff4-37f27be5ae03 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:06.967612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef462604-ca83-47eb-aff4-37f27be5ae03 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:06.969174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef462604-ca83-47eb-aff4-37f27be5ae03 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:06.971586 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef462604-ca83-47eb-aff4-37f27be5ae03 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:06.978190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:07.003965 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 27/55] 10.222.0.61 () {60 vars in 1140 bytes} [Thu Jul 14 10:00:06 2022] POST /identity/v3/projects => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:00:07.077679 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32acce2-83a6-43a9-a677-72d625e70199 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:07.123240 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc81e3-41c4-497f-b005-7c7fe739db4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:07.124061 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc81e3-41c4-497f-b005-7c7fe739db4b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:07.124345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc81e3-41c4-497f-b005-7c7fe739db4b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:07.124619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc81e3-41c4-497f-b005-7c7fe739db4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:07.125897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc81e3-41c4-497f-b005-7c7fe739db4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:07.127358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc81e3-41c4-497f-b005-7c7fe739db4b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:07.129048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 29/56] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:00:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:07.135563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef462604-ca83-47eb-aff4-37f27be5ae03 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:07.238661 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef462604-ca83-47eb-aff4-37f27be5ae03 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:07.272797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a93463-4a2f-46f6-8f7f-75517e78deca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:07.273493 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a93463-4a2f-46f6-8f7f-75517e78deca None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:07.273778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a93463-4a2f-46f6-8f7f-75517e78deca None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:07.274072 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a93463-4a2f-46f6-8f7f-75517e78deca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:07.274915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a93463-4a2f-46f6-8f7f-75517e78deca None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.domain.id=default, user.id=e054eebdda2a494f804b604ef19ea604) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:07.275225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a93463-4a2f-46f6-8f7f-75517e78deca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:07.276676 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a93463-4a2f-46f6-8f7f-75517e78deca None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:07.282642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 28/57] 10.222.0.61 () {56 vars in 1322 bytes} [Thu Jul 14 10:00:07 2022] GET /identity/v3/role_assignments?user.id=e054eebdda2a494f804b604ef19ea604&scope.domain.id=default&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:09.064404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b13e5-5eef-4ca8-ba0e-56057cb9e734 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:09.064951 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b13e5-5eef-4ca8-ba0e-56057cb9e734 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:09.064951 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69b13e5-5eef-4ca8-ba0e-56057cb9e734 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:09.066349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 30/58] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:09.071962 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1459f-6355-4dc3-abba-60fca47ae664 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:09.072359 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1459f-6355-4dc3-abba-60fca47ae664 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:09.072517 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b1459f-6355-4dc3-abba-60fca47ae664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:09.090309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-16b1459f-6355-4dc3-abba-60fca47ae664 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:09.091509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-16b1459f-6355-4dc3-abba-60fca47ae664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:09.138522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:09.158667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 29/59] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:09.183508 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b13e5-5eef-4ca8-ba0e-56057cb9e734 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:09.253956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:09.283670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69b13e5-5eef-4ca8-ba0e-56057cb9e734 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:09.319113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fccaea-27e9-44c0-b026-ada68324b0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:09.320392 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fccaea-27e9-44c0-b026-ada68324b0fd None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:09.320676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fccaea-27e9-44c0-b026-ada68324b0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:09.320978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fccaea-27e9-44c0-b026-ada68324b0fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:09.322164 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fccaea-27e9-44c0-b026-ada68324b0fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:09.323603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fccaea-27e9-44c0-b026-ada68324b0fd None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:09.325580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 31/60] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:00:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:09.331020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1459f-6355-4dc3-abba-60fca47ae664 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:09.427742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b1459f-6355-4dc3-abba-60fca47ae664 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:09.461793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712b698-6114-4220-974b-ed6ed377133b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:09.462457 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712b698-6114-4220-974b-ed6ed377133b None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:09.462745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712b698-6114-4220-974b-ed6ed377133b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:09.463028 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712b698-6114-4220-974b-ed6ed377133b None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:09.463998 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712b698-6114-4220-974b-ed6ed377133b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:09.465468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712b698-6114-4220-974b-ed6ed377133b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:09.490194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 30/61] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:00:09 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:00:12.641689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ed024-fa87-4778-b06d-230b27737cff None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:12.642690 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ed024-fa87-4778-b06d-230b27737cff None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:12.643180 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ed024-fa87-4778-b06d-230b27737cff None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:12.644779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 32/62] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:12.650224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1e1b0-3dcd-4af8-8776-bfd92d3c7e2f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:12.651092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1e1b0-3dcd-4af8-8776-bfd92d3c7e2f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:12.651584 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1e1b0-3dcd-4af8-8776-bfd92d3c7e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:12.678220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-cfe1e1b0-3dcd-4af8-8776-bfd92d3c7e2f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:12.679740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cfe1e1b0-3dcd-4af8-8776-bfd92d3c7e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:12.710806 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:12.759289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 31/63] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:12.806449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ed024-fa87-4778-b06d-230b27737cff None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:12.904661 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80b2d6-fb97-4442-b7e4-363eb0b6214e None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:12.904984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80b2d6-fb97-4442-b7e4-363eb0b6214e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:12.905257 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80b2d6-fb97-4442-b7e4-363eb0b6214e None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:12.906767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 32/64] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:12.912253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077c539-a50e-48ff-abf6-8026c25799f9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:12.912539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077c539-a50e-48ff-abf6-8026c25799f9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:12.912821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077c539-a50e-48ff-abf6-8026c25799f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:12.930204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ed024-fa87-4778-b06d-230b27737cff None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:12.932180 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3077c539-a50e-48ff-abf6-8026c25799f9 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:12.933398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3077c539-a50e-48ff-abf6-8026c25799f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:12.966182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa8d4d-af9f-4e67-bda6-f691d1c8d282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:12.966976 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa8d4d-af9f-4e67-bda6-f691d1c8d282 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:12.967263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa8d4d-af9f-4e67-bda6-f691d1c8d282 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:12.967538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa8d4d-af9f-4e67-bda6-f691d1c8d282 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:12.976670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:12.981783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa8d4d-af9f-4e67-bda6-f691d1c8d282 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-d6aa8d4d-af9f-4e67-bda6-f691d1c8d282 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:12.990333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:12.991364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:12.992373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:12.992373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:12.992373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:12.992373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:12.992373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:12.992373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:12.992373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:12.992373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:12.992373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:12.992373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:12.992373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:12.992373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:12.994408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 33/65] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:12.999102 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f48f7f-7e0b-49b0-88db-b8e49ee930f4 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:12.999268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f48f7f-7e0b-49b0-88db-b8e49ee930f4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:12.999783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f48f7f-7e0b-49b0-88db-b8e49ee930f4 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:13.001525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 34/66] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:13.006137 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d404f7-763b-476a-9432-54156e808327 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:13.006431 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d404f7-763b-476a-9432-54156e808327 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:13.006712 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d404f7-763b-476a-9432-54156e808327 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:13.007940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 35/67] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:13.009848 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 33/68] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:13.011035 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63286c-960b-4f4f-a1cb-0d4a459a2a01 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:13.011312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63286c-960b-4f4f-a1cb-0d4a459a2a01 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:13.011592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63286c-960b-4f4f-a1cb-0d4a459a2a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:13.012523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d404f7-763b-476a-9432-54156e808327 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:13.033481 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-1d63286c-960b-4f4f-a1cb-0d4a459a2a01 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:13.034913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1d63286c-960b-4f4f-a1cb-0d4a459a2a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:13.105475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 34/69] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:13.107119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d694a-c266-4d89-91e3-b16ae99c1ddc None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:13.107470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d694a-c266-4d89-91e3-b16ae99c1ddc None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:13.107819 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d694a-c266-4d89-91e3-b16ae99c1ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:13.124184 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-df6d694a-c266-4d89-91e3-b16ae99c1ddc None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:13.125612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-df6d694a-c266-4d89-91e3-b16ae99c1ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:13.136873 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d404f7-763b-476a-9432-54156e808327 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:13.150742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:13.175086 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972ca4f-f6fc-4b30-b569-e3f31295e9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:13.175687 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972ca4f-f6fc-4b30-b569-e3f31295e9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:13.175975 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972ca4f-f6fc-4b30-b569-e3f31295e9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:13.176256 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9972ca4f-f6fc-4b30-b569-e3f31295e9a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:13.177416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972ca4f-f6fc-4b30-b569-e3f31295e9a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:13.177748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972ca4f-f6fc-4b30-b569-e3f31295e9a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:13.179127 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9972ca4f-f6fc-4b30-b569-e3f31295e9a7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:13.187940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 36/70] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:13.188749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972ca4f-f6fc-4b30-b569-e3f31295e9a7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:13.208434 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 35/71] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:13.209624 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d694a-c266-4d89-91e3-b16ae99c1ddc None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:13.267425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:13.305585 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9972ca4f-f6fc-4b30-b569-e3f31295e9a7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:13.308537 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d694a-c266-4d89-91e3-b16ae99c1ddc None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:13.339584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542de7fd-021d-40c8-b6bc-53cdec64111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:13.340374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542de7fd-021d-40c8-b6bc-53cdec64111e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:13.340654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542de7fd-021d-40c8-b6bc-53cdec64111e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:13.340945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542de7fd-021d-40c8-b6bc-53cdec64111e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:13.343283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786610d5-3b5d-4e56-9fc9-1e5bb39e033f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:13.344086 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786610d5-3b5d-4e56-9fc9-1e5bb39e033f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:13.344370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786610d5-3b5d-4e56-9fc9-1e5bb39e033f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:13.344643 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786610d5-3b5d-4e56-9fc9-1e5bb39e033f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:13.350919 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542de7fd-021d-40c8-b6bc-53cdec64111e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:13.354728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786610d5-3b5d-4e56-9fc9-1e5bb39e033f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-542de7fd-021d-40c8-b6bc-53cdec64111e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:13.358577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:13.359566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:13.360331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:13.360331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:13.360331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:13.360331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:13.360331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:13.360331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:13.360331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:13.360331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:13.360331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:13.360331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:13.360331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:13.360331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:13.361212 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 37/72] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:13.361515 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542de7fd-021d-40c8-b6bc-53cdec64111e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-786610d5-3b5d-4e56-9fc9-1e5bb39e033f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:13.362725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:13.363527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:13.370374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:13.370374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:13.370374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:13.370374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:13.370374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:13.370374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:13.370374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:13.370374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:13.370374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:13.370374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:13.370374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:13.370374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:13.370374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 36/73] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:00:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:13.370374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b5780-5741-4f90-b998-9dad9021839c None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:13.371896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b5780-5741-4f90-b998-9dad9021839c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:13.371896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b5780-5741-4f90-b998-9dad9021839c None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:13.371896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 37/74] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:13.371896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b5780-5741-4f90-b998-9dad9021839c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:13.388460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:13.462011 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542de7fd-021d-40c8-b6bc-53cdec64111e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:13.471099 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b5780-5741-4f90-b998-9dad9021839c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:13.483201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:13.495820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd26b95-6196-4b4d-a52c-a95123f755d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:13.496699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd26b95-6196-4b4d-a52c-a95123f755d9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:13.497021 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd26b95-6196-4b4d-a52c-a95123f755d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:13.497303 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd26b95-6196-4b4d-a52c-a95123f755d9 None admin] PATH_INFO: `/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:13.505203 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356484b1-8790-434f-b9fa-2cbcf20fa6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:13.505991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356484b1-8790-434f-b9fa-2cbcf20fa6ef None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:13.506271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356484b1-8790-434f-b9fa-2cbcf20fa6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:13.506545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356484b1-8790-434f-b9fa-2cbcf20fa6ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:13.512803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd26b95-6196-4b4d-a52c-a95123f755d9 None admin] RBAC: Authorizing `identity:get_user(user_id=aabb9fdd0abd4b0e9c39fa9e7814c733)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:13.514298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd26b95-6196-4b4d-a52c-a95123f755d9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:13.515965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 38/75] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:13 2022] GET /identity/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:13.516621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356484b1-8790-434f-b9fa-2cbcf20fa6ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:13.516748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd26b95-6196-4b4d-a52c-a95123f755d9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-356484b1-8790-434f-b9fa-2cbcf20fa6ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:13.524322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:13.525439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:13.526360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:13.526360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:13.526360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:13.526360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:13.526360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:13.526360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:13.526360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:13.526360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:13.526360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:13.526360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:13.526360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:13.526360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:13.526360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 38/76] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:00:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:13.526360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356484b1-8790-434f-b9fa-2cbcf20fa6ef None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:13.603363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:13.617424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd26b95-6196-4b4d-a52c-a95123f755d9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:13.625046 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356484b1-8790-434f-b9fa-2cbcf20fa6ef None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:13.650778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3085e21-8e0f-4ccc-8ed4-6e40d9cfd734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:13.651463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3085e21-8e0f-4ccc-8ed4-6e40d9cfd734 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:13.651748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3085e21-8e0f-4ccc-8ed4-6e40d9cfd734 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:13.652030 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3085e21-8e0f-4ccc-8ed4-6e40d9cfd734 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:13.652794 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3085e21-8e0f-4ccc-8ed4-6e40d9cfd734 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:13.653139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3085e21-8e0f-4ccc-8ed4-6e40d9cfd734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:13.654538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3085e21-8e0f-4ccc-8ed4-6e40d9cfd734 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:13.658954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79970c4-95ef-4e6a-8130-2202627dafd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:13.659627 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79970c4-95ef-4e6a-8130-2202627dafd7 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:13.659905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79970c4-95ef-4e6a-8130-2202627dafd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:13.660180 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79970c4-95ef-4e6a-8130-2202627dafd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:13.660981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79970c4-95ef-4e6a-8130-2202627dafd7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:13.661290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79970c4-95ef-4e6a-8130-2202627dafd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:13.662664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79970c4-95ef-4e6a-8130-2202627dafd7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:13.669254 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 39/77] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:13.669254 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c192e2-a023-4f0b-9279-05f8ee1634d4 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:13.669254 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c192e2-a023-4f0b-9279-05f8ee1634d4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:13.669254 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c192e2-a023-4f0b-9279-05f8ee1634d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:13.670976 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 39/78] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:00:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:13.671773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79970c4-95ef-4e6a-8130-2202627dafd7 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:13.684725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-b2c192e2-a023-4f0b-9279-05f8ee1634d4 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:13.686386 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b2c192e2-a023-4f0b-9279-05f8ee1634d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:13.721283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:13.752746 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 40/79] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:13.763530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c192e2-a023-4f0b-9279-05f8ee1634d4 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:13.773112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79970c4-95ef-4e6a-8130-2202627dafd7 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:13.807224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204a8eb-c0fc-4022-b36f-db92ce5c381c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:13.807873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204a8eb-c0fc-4022-b36f-db92ce5c381c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:13.808158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204a8eb-c0fc-4022-b36f-db92ce5c381c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:13.808438 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204a8eb-c0fc-4022-b36f-db92ce5c381c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:13.818905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:13.861330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c192e2-a023-4f0b-9279-05f8ee1634d4 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:13.875319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204a8eb-c0fc-4022-b36f-db92ce5c381c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:13.876785 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204a8eb-c0fc-4022-b36f-db92ce5c381c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:13.895142 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a83b07-b69c-4180-9064-10b6ff3a30f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:13.895816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a83b07-b69c-4180-9064-10b6ff3a30f9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:13.896103 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a83b07-b69c-4180-9064-10b6ff3a30f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:13.896380 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a83b07-b69c-4180-9064-10b6ff3a30f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:13.897178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a83b07-b69c-4180-9064-10b6ff3a30f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:13.897505 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a83b07-b69c-4180-9064-10b6ff3a30f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:13.898866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a83b07-b69c-4180-9064-10b6ff3a30f9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:13.906742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 41/80] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:13.910999 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a83b07-b69c-4180-9064-10b6ff3a30f9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:13.945576 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:13.976990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 40/81] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:13.977847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204a8eb-c0fc-4022-b36f-db92ce5c381c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:14.013721 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a83b07-b69c-4180-9064-10b6ff3a30f9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:14.020497 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:14.048194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7aa529-b4d1-4d82-8437-49017d6cdaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:14.049118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7aa529-b4d1-4d82-8437-49017d6cdaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:14.049408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7aa529-b4d1-4d82-8437-49017d6cdaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:14.049686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7aa529-b4d1-4d82-8437-49017d6cdaf7 None admin] PATH_INFO: `/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:14.050943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7aa529-b4d1-4d82-8437-49017d6cdaf7 None admin] RBAC: Authorizing `identity:get_user(user_id=aabb9fdd0abd4b0e9c39fa9e7814c733)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:14.052893 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7aa529-b4d1-4d82-8437-49017d6cdaf7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:14.054568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 42/82] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:13 2022] GET /identity/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:14.056711 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7aa529-b4d1-4d82-8437-49017d6cdaf7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:14.079132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204a8eb-c0fc-4022-b36f-db92ce5c381c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:14.112633 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7f27d-d4c3-477a-980d-b85913a7abd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:14.114161 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7f27d-d4c3-477a-980d-b85913a7abd6 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:14.114458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7f27d-d4c3-477a-980d-b85913a7abd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:14.114735 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b7f27d-d4c3-477a-980d-b85913a7abd6 None admin] PATH_INFO: `/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:14.115943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7f27d-d4c3-477a-980d-b85913a7abd6 None admin] RBAC: Authorizing `identity:get_user(user_id=aabb9fdd0abd4b0e9c39fa9e7814c733)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:14.117438 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b7f27d-d4c3-477a-980d-b85913a7abd6 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:14.119073 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 41/83] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:13 2022] GET /identity/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:14.119929 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7f27d-d4c3-477a-980d-b85913a7abd6 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:14.161583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7aa529-b4d1-4d82-8437-49017d6cdaf7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:14.163092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:14.195749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57339076-a304-4d35-8b4b-1c7239fd101a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:14.196567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57339076-a304-4d35-8b4b-1c7239fd101a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:14.196853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57339076-a304-4d35-8b4b-1c7239fd101a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:14.197161 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57339076-a304-4d35-8b4b-1c7239fd101a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:14.198354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57339076-a304-4d35-8b4b-1c7239fd101a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:14.199815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57339076-a304-4d35-8b4b-1c7239fd101a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:14.201495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 43/84] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:00:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:14.202258 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57339076-a304-4d35-8b4b-1c7239fd101a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:14.224458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b7f27d-d4c3-477a-980d-b85913a7abd6 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:14.233944 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:14.261379 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ad8e9-2363-4e52-922f-fc7930be4397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:14.262261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ad8e9-2363-4e52-922f-fc7930be4397 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:14.262548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ad8e9-2363-4e52-922f-fc7930be4397 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:14.262828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ad8e9-2363-4e52-922f-fc7930be4397 None admin] PATH_INFO: `/v3/users/e054eebdda2a494f804b604ef19ea604` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:14.264030 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ad8e9-2363-4e52-922f-fc7930be4397 None admin] RBAC: Authorizing `identity:get_user(user_id=e054eebdda2a494f804b604ef19ea604)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:14.265516 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ad8e9-2363-4e52-922f-fc7930be4397 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:14.267156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 42/85] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:14 2022] GET /identity/v3/users/e054eebdda2a494f804b604ef19ea604 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:14.267923 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ad8e9-2363-4e52-922f-fc7930be4397 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:14.309212 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57339076-a304-4d35-8b4b-1c7239fd101a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:14.343801 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cf874-88af-4cc4-a140-0bebc3fbc2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:14.344667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cf874-88af-4cc4-a140-0bebc3fbc2cb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:14.344986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cf874-88af-4cc4-a140-0bebc3fbc2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:14.345279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540cf874-88af-4cc4-a140-0bebc3fbc2cb None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:14.346473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540cf874-88af-4cc4-a140-0bebc3fbc2cb None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:14.348803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540cf874-88af-4cc4-a140-0bebc3fbc2cb None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:14.350552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 44/86] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:14 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:14.351314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cf874-88af-4cc4-a140-0bebc3fbc2cb None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:14.375047 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ad8e9-2363-4e52-922f-fc7930be4397 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:14.390564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:14.408795 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91090332-da32-4e6f-a70f-7e46a20384ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:14.409476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91090332-da32-4e6f-a70f-7e46a20384ca None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:14.409763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91090332-da32-4e6f-a70f-7e46a20384ca None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:14.410577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91090332-da32-4e6f-a70f-7e46a20384ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:14.451419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540cf874-88af-4cc4-a140-0bebc3fbc2cb None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:14.453925 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:14.477880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91090332-da32-4e6f-a70f-7e46a20384ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:14.479359 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91090332-da32-4e6f-a70f-7e46a20384ca None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:14.485546 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f1d82-563b-4852-a388-bc5647d6cb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:14.486143 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f1d82-563b-4852-a388-bc5647d6cb71 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:14.486428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f1d82-563b-4852-a388-bc5647d6cb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:14.486702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f1d82-563b-4852-a388-bc5647d6cb71 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:14.552479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f1d82-563b-4852-a388-bc5647d6cb71 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:14.553960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f1d82-563b-4852-a388-bc5647d6cb71 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:14.577549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 43/87] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:14.578366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91090332-da32-4e6f-a70f-7e46a20384ca None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:14.592808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:14.651035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 45/88] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:14.651871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f1d82-563b-4852-a388-bc5647d6cb71 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:14.657666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:14.679922 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91090332-da32-4e6f-a70f-7e46a20384ca None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:14.714148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2831a74-1950-4752-b486-f6d8acf07595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:14.714816 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2831a74-1950-4752-b486-f6d8acf07595 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:14.715104 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2831a74-1950-4752-b486-f6d8acf07595 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:14.715388 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2831a74-1950-4752-b486-f6d8acf07595 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:14.716115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2831a74-1950-4752-b486-f6d8acf07595 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:14.717630 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2831a74-1950-4752-b486-f6d8acf07595 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:14.736590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 44/89] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:00:14 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:00:14.737497 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2831a74-1950-4752-b486-f6d8acf07595 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:14.767016 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f1d82-563b-4852-a388-bc5647d6cb71 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:14.803458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c119402-76ae-4ec9-812c-4681f1205e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:14.804132 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c119402-76ae-4ec9-812c-4681f1205e56 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:14.804419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c119402-76ae-4ec9-812c-4681f1205e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:14.804699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c119402-76ae-4ec9-812c-4681f1205e56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:14.814885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:14.855204 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2831a74-1950-4752-b486-f6d8acf07595 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:14.884970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c119402-76ae-4ec9-812c-4681f1205e56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:14.884970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c119402-76ae-4ec9-812c-4681f1205e56 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:14.891843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:14.895957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ebe35-ea0d-4c91-8b4e-36629e4b9db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:14.896985 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ebe35-ea0d-4c91-8b4e-36629e4b9db0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:14.898037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ebe35-ea0d-4c91-8b4e-36629e4b9db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:14.898315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ebe35-ea0d-4c91-8b4e-36629e4b9db0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:14.899957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ebe35-ea0d-4c91-8b4e-36629e4b9db0 None admin] RBAC: Adding query filter params (role.id=3d9688bbbc15422cbfc59ded082fbe41, scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c, user.id=aabb9fdd0abd4b0e9c39fa9e7814c733) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:14.900506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ebe35-ea0d-4c91-8b4e-36629e4b9db0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:14.902267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ebe35-ea0d-4c91-8b4e-36629e4b9db0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:14.908739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 45/90] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:14 2022] GET /identity/v3/role_assignments?user.id=aabb9fdd0abd4b0e9c39fa9e7814c733&scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c&role.id=3d9688bbbc15422cbfc59ded082fbe41 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:14.909687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ebe35-ea0d-4c91-8b4e-36629e4b9db0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:15.020352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 46/91] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:15.024998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c119402-76ae-4ec9-812c-4681f1205e56 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:15.058284 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ebe35-ea0d-4c91-8b4e-36629e4b9db0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:15.074863 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:15.103332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7546fb6-70bb-45ba-b3b6-d74254d95b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:15.104149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7546fb6-70bb-45ba-b3b6-d74254d95b48 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:15.104438 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7546fb6-70bb-45ba-b3b6-d74254d95b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:15.104739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7546fb6-70bb-45ba-b3b6-d74254d95b48 None admin] PATH_INFO: `/v3/projects/2f57e9f519db4d1c9d4e343df51aa79e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:15.106100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7546fb6-70bb-45ba-b3b6-d74254d95b48 None admin] RBAC: Authorizing `identity:get_project(project_id=2f57e9f519db4d1c9d4e343df51aa79e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:15.107560 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7546fb6-70bb-45ba-b3b6-d74254d95b48 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:15.109550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 46/92] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:14 2022] GET /identity/v3/projects/2f57e9f519db4d1c9d4e343df51aa79e => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:15.110164 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7546fb6-70bb-45ba-b3b6-d74254d95b48 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:15.153447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c119402-76ae-4ec9-812c-4681f1205e56 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:15.193344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:15.194929 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61fe47-64a1-4b50-acb6-b3781e77c78b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:15.195815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61fe47-64a1-4b50-acb6-b3781e77c78b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:15.196094 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61fe47-64a1-4b50-acb6-b3781e77c78b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:15.196366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a61fe47-64a1-4b50-acb6-b3781e77c78b None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:15.197618 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61fe47-64a1-4b50-acb6-b3781e77c78b None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:15.199051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a61fe47-64a1-4b50-acb6-b3781e77c78b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:15.200768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 47/93] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:15 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:15.201570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61fe47-64a1-4b50-acb6-b3781e77c78b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:15.250316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7546fb6-70bb-45ba-b3b6-d74254d95b48 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:15.293024 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7304b-6320-4b2e-b475-ae7b8fe8530e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:15.294688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7304b-6320-4b2e-b475-ae7b8fe8530e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:15.295174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7304b-6320-4b2e-b475-ae7b8fe8530e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:15.295664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc7304b-6320-4b2e-b475-ae7b8fe8530e None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:15.297173 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7304b-6320-4b2e-b475-ae7b8fe8530e None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:15.299031 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc7304b-6320-4b2e-b475-ae7b8fe8530e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:15.301195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 47/94] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:15 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:15.302371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7304b-6320-4b2e-b475-ae7b8fe8530e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:15.375711 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a61fe47-64a1-4b50-acb6-b3781e77c78b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:15.383726 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:15.424471 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc7304b-6320-4b2e-b475-ae7b8fe8530e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:15.434993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643adbd-54cd-41c5-994c-cddcf466e256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:15.435694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643adbd-54cd-41c5-994c-cddcf466e256 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:15.435945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643adbd-54cd-41c5-994c-cddcf466e256 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:15.436218 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643adbd-54cd-41c5-994c-cddcf466e256 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:15.437091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643adbd-54cd-41c5-994c-cddcf466e256 None admin] RBAC: Adding query filter params (role.id=3d9688bbbc15422cbfc59ded082fbe41, scope.project.id=2f57e9f519db4d1c9d4e343df51aa79e, user.id=aabb9fdd0abd4b0e9c39fa9e7814c733) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:15.437404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643adbd-54cd-41c5-994c-cddcf466e256 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:15.438848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643adbd-54cd-41c5-994c-cddcf466e256 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:15.456605 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 48/95] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:15 2022] GET /identity/v3/role_assignments?user.id=aabb9fdd0abd4b0e9c39fa9e7814c733&scope.project.id=2f57e9f519db4d1c9d4e343df51aa79e&role.id=3d9688bbbc15422cbfc59ded082fbe41 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:15.461225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643adbd-54cd-41c5-994c-cddcf466e256 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:15.466995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:15.473700 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c059cb-74b6-462c-8a8f-3da674fceb73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:15.474734 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c059cb-74b6-462c-8a8f-3da674fceb73 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:15.475237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c059cb-74b6-462c-8a8f-3da674fceb73 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:15.475788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c059cb-74b6-462c-8a8f-3da674fceb73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:15.476894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c059cb-74b6-462c-8a8f-3da674fceb73 None admin] RBAC: Adding query filter params (role.id=f4e1d6f34f674f539ea7f2564646c0a5, scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c, user.id=aabb9fdd0abd4b0e9c39fa9e7814c733) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:15.481758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c059cb-74b6-462c-8a8f-3da674fceb73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:15.483621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c059cb-74b6-462c-8a8f-3da674fceb73 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:15.493959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 48/96] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:15 2022] GET /identity/v3/role_assignments?user.id=aabb9fdd0abd4b0e9c39fa9e7814c733&scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c&role.id=f4e1d6f34f674f539ea7f2564646c0a5 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:15.606901 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643adbd-54cd-41c5-994c-cddcf466e256 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:15.645169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764a6052-f316-40eb-bccf-cf3bf42b9877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:15.645848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a6052-f316-40eb-bccf-cf3bf42b9877 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:15.646131 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a6052-f316-40eb-bccf-cf3bf42b9877 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:15.646407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764a6052-f316-40eb-bccf-cf3bf42b9877 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:15.647263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a6052-f316-40eb-bccf-cf3bf42b9877 None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c, user.id=e054eebdda2a494f804b604ef19ea604) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:15.647582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a6052-f316-40eb-bccf-cf3bf42b9877 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:15.649087 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764a6052-f316-40eb-bccf-cf3bf42b9877 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:15.655726 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 49/97] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:15 2022] GET /identity/v3/role_assignments?user.id=e054eebdda2a494f804b604ef19ea604&scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:17.431687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9fd84-e6d9-4d44-8668-68b2a03c20d5 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:17.432180 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9fd84-e6d9-4d44-8668-68b2a03c20d5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:17.432251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9fd84-e6d9-4d44-8668-68b2a03c20d5 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:17.433678 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 49/98] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:17.439113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821ff3c-e47b-44ad-97c9-4004ced7ab86 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:17.441290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821ff3c-e47b-44ad-97c9-4004ced7ab86 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:17.441627 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821ff3c-e47b-44ad-97c9-4004ced7ab86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:17.468623 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a821ff3c-e47b-44ad-97c9-4004ced7ab86 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:17.469885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a821ff3c-e47b-44ad-97c9-4004ced7ab86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:17.504356 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:17.560675 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 50/99] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:17.599667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9fd84-e6d9-4d44-8668-68b2a03c20d5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:17.726587 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9fd84-e6d9-4d44-8668-68b2a03c20d5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:17.772104 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b994eda-6581-46a3-a89e-5e6cca41f80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:17.772950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994eda-6581-46a3-a89e-5e6cca41f80c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:17.773252 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994eda-6581-46a3-a89e-5e6cca41f80c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:17.773528 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b994eda-6581-46a3-a89e-5e6cca41f80c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:17.786647 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b994eda-6581-46a3-a89e-5e6cca41f80c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-8b994eda-6581-46a3-a89e-5e6cca41f80c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:17.794396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:17.795253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:17.796017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:17.796017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:17.796017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:17.796017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:17.796017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:17.796017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:17.796017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:17.796017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:17.796017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:17.796017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:17.796017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:17.796017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:17.796017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 50/100] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:17.802909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821ff3c-e47b-44ad-97c9-4004ced7ab86 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:17.817736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d293123-9f78-457a-a333-3d6dbfe9cd54 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:17.818005 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d293123-9f78-457a-a333-3d6dbfe9cd54 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:17.818284 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d293123-9f78-457a-a333-3d6dbfe9cd54 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:17.819658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 51/101] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:00:17.825615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69074f-fcd1-4f60-bc03-068d46f0ba3e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:17.825901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69074f-fcd1-4f60-bc03-068d46f0ba3e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:17.826181 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69074f-fcd1-4f60-bc03-068d46f0ba3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:17.855288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-5e69074f-fcd1-4f60-bc03-068d46f0ba3e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:17.856654 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5e69074f-fcd1-4f60-bc03-068d46f0ba3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:17.972507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821ff3c-e47b-44ad-97c9-4004ced7ab86 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:17.980059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:18.020967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 52/102] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:18.022372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381eace-0d73-441e-9289-11cdb1c7377f None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:18.022602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381eace-0d73-441e-9289-11cdb1c7377f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:18.022938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381eace-0d73-441e-9289-11cdb1c7377f None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:18.024262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 53/103] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:18.030398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f978fea-a531-4c4a-b43b-2c4a1758375a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:18.030686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f978fea-a531-4c4a-b43b-2c4a1758375a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:18.030963 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f978fea-a531-4c4a-b43b-2c4a1758375a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:18.057157 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662a431-8094-42df-bd82-fb1b7bad7c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:18.057850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662a431-8094-42df-bd82-fb1b7bad7c98 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:18.058135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662a431-8094-42df-bd82-fb1b7bad7c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:18.058414 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3662a431-8094-42df-bd82-fb1b7bad7c98 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:18.059187 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662a431-8094-42df-bd82-fb1b7bad7c98 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:18.059499 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662a431-8094-42df-bd82-fb1b7bad7c98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:18.060904 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3662a431-8094-42df-bd82-fb1b7bad7c98 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:18.074203 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-1f978fea-a531-4c4a-b43b-2c4a1758375a None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:18.076959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1f978fea-a531-4c4a-b43b-2c4a1758375a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:18.081242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 51/104] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:18.088946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662a431-8094-42df-bd82-fb1b7bad7c98 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:18.117477 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:18.149312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 54/105] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:18.150501 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f978fea-a531-4c4a-b43b-2c4a1758375a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:18.192264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3662a431-8094-42df-bd82-fb1b7bad7c98 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:18.230421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9059bb-3f28-48b5-80b7-9fd579c0505a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:18.231573 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9059bb-3f28-48b5-80b7-9fd579c0505a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:18.232055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9059bb-3f28-48b5-80b7-9fd579c0505a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:18.232522 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9059bb-3f28-48b5-80b7-9fd579c0505a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:18.234001 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9059bb-3f28-48b5-80b7-9fd579c0505a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:18.241582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9059bb-3f28-48b5-80b7-9fd579c0505a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:18.243619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 52/106] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:00:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:18.250036 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9059bb-3f28-48b5-80b7-9fd579c0505a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:18.259182 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f978fea-a531-4c4a-b43b-2c4a1758375a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:18.277180 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:18.296740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b96d8e-6283-4801-8c03-be8519e6eb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:18.297679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b96d8e-6283-4801-8c03-be8519e6eb7b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:18.297969 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b96d8e-6283-4801-8c03-be8519e6eb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:18.298304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b96d8e-6283-4801-8c03-be8519e6eb7b None admin] PATH_INFO: `/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:18.300250 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b96d8e-6283-4801-8c03-be8519e6eb7b None admin] RBAC: Authorizing `identity:get_user(user_id=aabb9fdd0abd4b0e9c39fa9e7814c733)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:18.301759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b96d8e-6283-4801-8c03-be8519e6eb7b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:18.303520 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 55/107] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:18 2022] GET /identity/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:18.308390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3110865-d1e1-4dad-b9a5-7a4a0895bded None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:18.308390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3110865-d1e1-4dad-b9a5-7a4a0895bded None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:18.308390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3110865-d1e1-4dad-b9a5-7a4a0895bded None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:18.308390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 56/108] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:18.308390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3110865-d1e1-4dad-b9a5-7a4a0895bded None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:18.354822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:18.362562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9059bb-3f28-48b5-80b7-9fd579c0505a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:18.397066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e9a00-b0b6-4c4e-baeb-bec28cda360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:18.398219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e9a00-b0b6-4c4e-baeb-bec28cda360d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:18.398697 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e9a00-b0b6-4c4e-baeb-bec28cda360d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:18.399165 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e9a00-b0b6-4c4e-baeb-bec28cda360d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:18.410761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e9a00-b0b6-4c4e-baeb-bec28cda360d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:18.418665 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3110865-d1e1-4dad-b9a5-7a4a0895bded None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-c59e9a00-b0b6-4c4e-baeb-bec28cda360d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:18.419139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:18.419988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:18.420767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:18.420767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:18.420767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:18.420767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:18.420767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:18.420767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:18.420767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:18.420767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:18.420767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:18.420767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:18.420767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:18.420767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:18.424142 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 53/109] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:18.425235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e9a00-b0b6-4c4e-baeb-bec28cda360d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:18.456344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16541c-da86-41a6-810b-26f862044e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:18.457041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16541c-da86-41a6-810b-26f862044e64 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:18.457284 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16541c-da86-41a6-810b-26f862044e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:18.457679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16541c-da86-41a6-810b-26f862044e64 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:18.458711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16541c-da86-41a6-810b-26f862044e64 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:18.460384 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16541c-da86-41a6-810b-26f862044e64 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:18.491112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 57/110] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Jul 14 10:00:18 2022] POST /identity/v3/projects => generated 302 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:00:18.492633 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aa7c2-f02a-444a-a574-573cd85d9aec None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:18.493323 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aa7c2-f02a-444a-a574-573cd85d9aec None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:18.493669 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67aa7c2-f02a-444a-a574-573cd85d9aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:18.512891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c67aa7c2-f02a-444a-a574-573cd85d9aec None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:18.514198 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c67aa7c2-f02a-444a-a574-573cd85d9aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:18.537291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e9a00-b0b6-4c4e-baeb-bec28cda360d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:18.549417 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:18.575616 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ae03f-2796-4c31-91b8-d39f031e0601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:18.576884 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ae03f-2796-4c31-91b8-d39f031e0601 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:18.577531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ae03f-2796-4c31-91b8-d39f031e0601 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:18.578016 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28ae03f-2796-4c31-91b8-d39f031e0601 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:18.579632 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ae03f-2796-4c31-91b8-d39f031e0601 None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:18.581434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28ae03f-2796-4c31-91b8-d39f031e0601 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:18.583881 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 54/111] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:18 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:18.585064 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ae03f-2796-4c31-91b8-d39f031e0601 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:18.600364 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 58/112] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:18.601567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aa7c2-f02a-444a-a574-573cd85d9aec None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:18.612611 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:18.696181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28ae03f-2796-4c31-91b8-d39f031e0601 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:18.723152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67aa7c2-f02a-444a-a574-573cd85d9aec None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:18.734636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf2258-85ef-4a5c-a686-151a28c78d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:18.735851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf2258-85ef-4a5c-a686-151a28c78d6a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:18.736379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf2258-85ef-4a5c-a686-151a28c78d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:18.736852 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bf2258-85ef-4a5c-a686-151a28c78d6a None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:18.737943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf2258-85ef-4a5c-a686-151a28c78d6a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:18.738468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf2258-85ef-4a5c-a686-151a28c78d6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:18.740102 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bf2258-85ef-4a5c-a686-151a28c78d6a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:18.749935 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 55/113] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:18.751066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf2258-85ef-4a5c-a686-151a28c78d6a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:18.760947 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e817486-b209-4745-b042-094414589192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:18.761829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e817486-b209-4745-b042-094414589192 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:18.762117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e817486-b209-4745-b042-094414589192 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:18.762398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e817486-b209-4745-b042-094414589192 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c/users/aabb9fdd0abd4b0e9c39fa9e7814c733/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:18.764653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e817486-b209-4745-b042-094414589192 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee948cfa1f0c4c65a69fc5204872db0c, user_id=aabb9fdd0abd4b0e9c39fa9e7814c733, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:18.766344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e817486-b209-4745-b042-094414589192 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:18.785200 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 59/114] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:00:18 2022] PUT /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c/users/aabb9fdd0abd4b0e9c39fa9e7814c733/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:00:18.786346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e817486-b209-4745-b042-094414589192 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:18.819871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:18.851703 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:18.869943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bf2258-85ef-4a5c-a686-151a28c78d6a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:18.911308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb960f24-479f-4449-bec6-0230c9a99481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:18.912475 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb960f24-479f-4449-bec6-0230c9a99481 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:18.912861 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb960f24-479f-4449-bec6-0230c9a99481 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:18.913311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb960f24-479f-4449-bec6-0230c9a99481 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:18.927827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb960f24-479f-4449-bec6-0230c9a99481 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:18.931371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e817486-b209-4745-b042-094414589192 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-cb960f24-479f-4449-bec6-0230c9a99481 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:18.937822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:18.939540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:18.940543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:18.940543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:18.940543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:18.940543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:18.940543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:18.940543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:18.940543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:18.940543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:18.940543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:18.940543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:18.940543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:18.940543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:18.942376 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 56/115] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:18.948398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb960f24-479f-4449-bec6-0230c9a99481 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:19.000140 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542e4e9-6333-45d0-a842-4df4be34792d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:19.001174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542e4e9-6333-45d0-a842-4df4be34792d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:19.001456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542e4e9-6333-45d0-a842-4df4be34792d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:19.001729 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542e4e9-6333-45d0-a842-4df4be34792d None admin] PATH_INFO: `/v3/users/57c664aabb3c487e88c8884d7ede941f` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:19.016890 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542e4e9-6333-45d0-a842-4df4be34792d None admin] RBAC: Authorizing `identity:get_user(user_id=57c664aabb3c487e88c8884d7ede941f)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:19.018543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542e4e9-6333-45d0-a842-4df4be34792d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:19.020556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 60/116] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:18 2022] GET /identity/v3/users/57c664aabb3c487e88c8884d7ede941f => generated 289 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:19.030464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542e4e9-6333-45d0-a842-4df4be34792d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:19.076071 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb960f24-479f-4449-bec6-0230c9a99481 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:19.114180 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aa2d00-7487-4e3f-8e2b-447269cef054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:19.114971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa2d00-7487-4e3f-8e2b-447269cef054 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:19.115270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa2d00-7487-4e3f-8e2b-447269cef054 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:19.115553 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aa2d00-7487-4e3f-8e2b-447269cef054 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:19.116323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa2d00-7487-4e3f-8e2b-447269cef054 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:19.116639 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa2d00-7487-4e3f-8e2b-447269cef054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:19.118600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aa2d00-7487-4e3f-8e2b-447269cef054 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:19.128233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 57/117] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:19.129717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7100f4-9d45-42b0-8db4-c21c4ec50c7f None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:19.130012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7100f4-9d45-42b0-8db4-c21c4ec50c7f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:19.130416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7100f4-9d45-42b0-8db4-c21c4ec50c7f None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:19.131926 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 58/118] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:19.133593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:19.133961 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7100f4-9d45-42b0-8db4-c21c4ec50c7f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:19.150765 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:19.159077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542e4e9-6333-45d0-a842-4df4be34792d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:19.201623 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea04d82-d264-4c4a-970d-7b2b8bcb780f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:19.202286 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04d82-d264-4c4a-970d-7b2b8bcb780f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:19.202570 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04d82-d264-4c4a-970d-7b2b8bcb780f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:19.202843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea04d82-d264-4c4a-970d-7b2b8bcb780f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:19.271680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7100f4-9d45-42b0-8db4-c21c4ec50c7f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:19.296602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea04d82-d264-4c4a-970d-7b2b8bcb780f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:19.299813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea04d82-d264-4c4a-970d-7b2b8bcb780f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:19.313843 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567df48-a14f-4142-94db-47ead47c18d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:19.314820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567df48-a14f-4142-94db-47ead47c18d7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:19.315175 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567df48-a14f-4142-94db-47ead47c18d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:19.315525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567df48-a14f-4142-94db-47ead47c18d7 None admin] PATH_INFO: `/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:19.316873 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567df48-a14f-4142-94db-47ead47c18d7 None admin] RBAC: Authorizing `identity:get_user(user_id=aabb9fdd0abd4b0e9c39fa9e7814c733)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:19.318483 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567df48-a14f-4142-94db-47ead47c18d7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:19.320273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 59/119] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:19 2022] GET /identity/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733 => generated 275 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:19.321649 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb9777-4564-4828-bee2-59f6af8fb840 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:19.322012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb9777-4564-4828-bee2-59f6af8fb840 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:19.322370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb9777-4564-4828-bee2-59f6af8fb840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:19.340514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-cafb9777-4564-4828-bee2-59f6af8fb840 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:19.341764 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-cafb9777-4564-4828-bee2-59f6af8fb840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:19.420601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 60/120] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:19.420601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5abaa3-6159-4e49-a265-742e68713ab2 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:19.420601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5abaa3-6159-4e49-a265-742e68713ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:19.420601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5abaa3-6159-4e49-a265-742e68713ab2 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:19.420601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 61/121] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:19.420601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5abaa3-6159-4e49-a265-742e68713ab2 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:19.426292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:19.429732 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:19.447052 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 61/122] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:00:19.455377 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369b673-365d-4b79-b4ae-df3602109f7d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:19.455847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369b673-365d-4b79-b4ae-df3602109f7d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:19.456265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369b673-365d-4b79-b4ae-df3602109f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:19.479152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9369b673-365d-4b79-b4ae-df3602109f7d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:19.481238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9369b673-365d-4b79-b4ae-df3602109f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:19.550897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5abaa3-6159-4e49-a265-742e68713ab2 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:19.558154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 62/123] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:19.559324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369b673-365d-4b79-b4ae-df3602109f7d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:19.609809 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c98b6-6e75-4bba-8541-3b287ff6ba0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:19.610703 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c98b6-6e75-4bba-8541-3b287ff6ba0c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:19.610988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c98b6-6e75-4bba-8541-3b287ff6ba0c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:19.611268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c98b6-6e75-4bba-8541-3b287ff6ba0c None admin] PATH_INFO: `/v3/projects/2f57e9f519db4d1c9d4e343df51aa79e` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:19.612493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c98b6-6e75-4bba-8541-3b287ff6ba0c None admin] RBAC: Authorizing `identity:get_project(project_id=2f57e9f519db4d1c9d4e343df51aa79e)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:19.614021 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c98b6-6e75-4bba-8541-3b287ff6ba0c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:19.616024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 62/124] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:19 2022] GET /identity/v3/projects/2f57e9f519db4d1c9d4e343df51aa79e => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:19.616694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c98b6-6e75-4bba-8541-3b287ff6ba0c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:19.668861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369b673-365d-4b79-b4ae-df3602109f7d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:19.684009 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:19.706371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9f9f-4f66-4c8d-be7d-3272960a356b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:19.707822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9f9f-4f66-4c8d-be7d-3272960a356b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:19.708300 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9f9f-4f66-4c8d-be7d-3272960a356b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:19.708770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904f9f9f-4f66-4c8d-be7d-3272960a356b None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:19.709312 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:19.710360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f9f9f-4f66-4c8d-be7d-3272960a356b None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:19.712020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904f9f9f-4f66-4c8d-be7d-3272960a356b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:19.713953 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 63/125] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:19 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:19.720242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9f9f-4f66-4c8d-be7d-3272960a356b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:19.728051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c98b6-6e75-4bba-8541-3b287ff6ba0c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:19.766261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f69ce6-379e-4818-b1c9-d62b10c547e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:19.767593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f69ce6-379e-4818-b1c9-d62b10c547e7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:19.767877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f69ce6-379e-4818-b1c9-d62b10c547e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:19.768152 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f69ce6-379e-4818-b1c9-d62b10c547e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:19.779707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f69ce6-379e-4818-b1c9-d62b10c547e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-e7f69ce6-379e-4818-b1c9-d62b10c547e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:19.788524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:19.789481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:19.790263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:19.790263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:19.790263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:19.790263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:19.790263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:19.790263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:19.790263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:19.790263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:19.790263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:19.790263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:19.790263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:19.790263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:19.790263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 63/126] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:00:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:19.790911 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f69ce6-379e-4818-b1c9-d62b10c547e7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:19.834720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904f9f9f-4f66-4c8d-be7d-3272960a356b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:19.869234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990678a9-f782-4408-b8fc-fbed1070e180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:19.870247 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990678a9-f782-4408-b8fc-fbed1070e180 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:19.870737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990678a9-f782-4408-b8fc-fbed1070e180 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:19.871214 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990678a9-f782-4408-b8fc-fbed1070e180 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:19.882683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990678a9-f782-4408-b8fc-fbed1070e180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-990678a9-f782-4408-b8fc-fbed1070e180 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:19.890915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:19.892345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:19.893177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:19.893177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:19.893177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:19.893177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:19.893177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:19.893177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:19.893177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:19.893177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:19.893177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:19.893177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:19.893177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:19.893177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:19.895199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 64/127] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:00:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:19.896138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990678a9-f782-4408-b8fc-fbed1070e180 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:19.906226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f69ce6-379e-4818-b1c9-d62b10c547e7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:19.941390 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:19.942705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:19.944219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee1d02-0ae9-423a-8ee6-22adf4000583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:19.945366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee1d02-0ae9-423a-8ee6-22adf4000583 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:19.945852 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee1d02-0ae9-423a-8ee6-22adf4000583 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:19.946304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee1d02-0ae9-423a-8ee6-22adf4000583 None admin] PATH_INFO: `/v3/projects/2f57e9f519db4d1c9d4e343df51aa79e/users/aabb9fdd0abd4b0e9c39fa9e7814c733/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:19.949009 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee1d02-0ae9-423a-8ee6-22adf4000583 None admin] RBAC: Authorizing `identity:create_grant(project_id=2f57e9f519db4d1c9d4e343df51aa79e, user_id=aabb9fdd0abd4b0e9c39fa9e7814c733, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:19.950836 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee1d02-0ae9-423a-8ee6-22adf4000583 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:19.972361 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 64/128] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:00:19 2022] PUT /identity/v3/projects/2f57e9f519db4d1c9d4e343df51aa79e/users/aabb9fdd0abd4b0e9c39fa9e7814c733/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:00:19.973929 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee1d02-0ae9-423a-8ee6-22adf4000583 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:20.012489 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990678a9-f782-4408-b8fc-fbed1070e180 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:20.050252 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d192bd-33e9-4967-ae11-7371797e478f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:20.050927 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d192bd-33e9-4967-ae11-7371797e478f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:20.051219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d192bd-33e9-4967-ae11-7371797e478f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:20.051502 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d192bd-33e9-4967-ae11-7371797e478f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:20.052357 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d192bd-33e9-4967-ae11-7371797e478f None admin] RBAC: Adding query filter params (role.id=73fb34f5dbe1404fa7ec015422951829, scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c, user.id=57c664aabb3c487e88c8884d7ede941f) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:20.052761 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d192bd-33e9-4967-ae11-7371797e478f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:20.054182 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d192bd-33e9-4967-ae11-7371797e478f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:20.060561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 65/129] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:19 2022] GET /identity/v3/role_assignments?user.id=57c664aabb3c487e88c8884d7ede941f&scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c&role.id=73fb34f5dbe1404fa7ec015422951829 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:20.066129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d192bd-33e9-4967-ae11-7371797e478f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:20.113564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee1d02-0ae9-423a-8ee6-22adf4000583 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:20.181902 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6855df88-897f-4bb5-9feb-73c3750b9a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:20.183036 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855df88-897f-4bb5-9feb-73c3750b9a90 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:20.183548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855df88-897f-4bb5-9feb-73c3750b9a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:20.184020 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6855df88-897f-4bb5-9feb-73c3750b9a90 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:20.185028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6855df88-897f-4bb5-9feb-73c3750b9a90 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:20.185639 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6855df88-897f-4bb5-9feb-73c3750b9a90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:20.187222 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6855df88-897f-4bb5-9feb-73c3750b9a90 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:20.205969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 65/130] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:00:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:20.212176 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6855df88-897f-4bb5-9feb-73c3750b9a90 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:20.270734 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d192bd-33e9-4967-ae11-7371797e478f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:20.293270 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:20.331243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce419277-cf35-430e-a533-50e8f36dcb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:20.332201 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce419277-cf35-430e-a533-50e8f36dcb0d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:20.332496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce419277-cf35-430e-a533-50e8f36dcb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:20.332781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce419277-cf35-430e-a533-50e8f36dcb0d None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:20.333866 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce419277-cf35-430e-a533-50e8f36dcb0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:20.334199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce419277-cf35-430e-a533-50e8f36dcb0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:20.336976 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce419277-cf35-430e-a533-50e8f36dcb0d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:20.346438 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 66/131] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:20.352654 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce419277-cf35-430e-a533-50e8f36dcb0d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:20.387746 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:20.459926 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce419277-cf35-430e-a533-50e8f36dcb0d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:20.469457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6855df88-897f-4bb5-9feb-73c3750b9a90 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:20.495454 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fe016-decb-4c2e-a0b8-0d864f21366b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:20.496399 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fe016-decb-4c2e-a0b8-0d864f21366b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:20.496684 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fe016-decb-4c2e-a0b8-0d864f21366b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:20.497048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95fe016-decb-4c2e-a0b8-0d864f21366b None admin] PATH_INFO: `/v3/users/e054eebdda2a494f804b604ef19ea604` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:20.498258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95fe016-decb-4c2e-a0b8-0d864f21366b None admin] RBAC: Authorizing `identity:get_user(user_id=e054eebdda2a494f804b604ef19ea604)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:20.499736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95fe016-decb-4c2e-a0b8-0d864f21366b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:20.501573 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 67/132] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:20 2022] GET /identity/v3/users/e054eebdda2a494f804b604ef19ea604 => generated 247 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:20.507113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fe016-decb-4c2e-a0b8-0d864f21366b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:20.534211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b792fcb-89a9-483c-8b58-bb34c5a70a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:20.535108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b792fcb-89a9-483c-8b58-bb34c5a70a9f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:20.535393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b792fcb-89a9-483c-8b58-bb34c5a70a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:20.535675 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b792fcb-89a9-483c-8b58-bb34c5a70a9f None admin] PATH_INFO: `/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:20.541675 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b792fcb-89a9-483c-8b58-bb34c5a70a9f None admin] RBAC: Authorizing `identity:get_user(user_id=aabb9fdd0abd4b0e9c39fa9e7814c733)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:20.543136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b792fcb-89a9-483c-8b58-bb34c5a70a9f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:20.545124 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 66/133] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:20 2022] GET /identity/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733 => generated 275 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:20.545771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:20.552351 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b792fcb-89a9-483c-8b58-bb34c5a70a9f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:20.662611 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95fe016-decb-4c2e-a0b8-0d864f21366b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:20.715657 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b792fcb-89a9-483c-8b58-bb34c5a70a9f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:20.739100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:20.751279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092aab0-d7c1-4e76-af0f-598eb8c5ebb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:20.752159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092aab0-d7c1-4e76-af0f-598eb8c5ebb0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:20.752447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092aab0-d7c1-4e76-af0f-598eb8c5ebb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:20.752728 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092aab0-d7c1-4e76-af0f-598eb8c5ebb0 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:20.753965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092aab0-d7c1-4e76-af0f-598eb8c5ebb0 None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:20.754091 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdeab8-14fb-4198-8bf1-77b7b0ae028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:20.754926 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdeab8-14fb-4198-8bf1-77b7b0ae028d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:20.755218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdeab8-14fb-4198-8bf1-77b7b0ae028d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:20.755429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092aab0-d7c1-4e76-af0f-598eb8c5ebb0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:20.755533 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdeab8-14fb-4198-8bf1-77b7b0ae028d None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:20.756689 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcdeab8-14fb-4198-8bf1-77b7b0ae028d None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:20.757498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 67/134] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:20 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:20.765134 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092aab0-d7c1-4e76-af0f-598eb8c5ebb0 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:20.766561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcdeab8-14fb-4198-8bf1-77b7b0ae028d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:20.768621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 68/135] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:20 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:20.777872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdeab8-14fb-4198-8bf1-77b7b0ae028d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:20.884449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092aab0-d7c1-4e76-af0f-598eb8c5ebb0 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:20.921300 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd374c9-8cc0-4ef5-8254-1b3ab93e0420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:20.922748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd374c9-8cc0-4ef5-8254-1b3ab93e0420 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:20.923141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd374c9-8cc0-4ef5-8254-1b3ab93e0420 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:20.923420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd374c9-8cc0-4ef5-8254-1b3ab93e0420 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c/users/aabb9fdd0abd4b0e9c39fa9e7814c733/roles/f4e1d6f34f674f539ea7f2564646c0a5` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:20.931210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd374c9-8cc0-4ef5-8254-1b3ab93e0420 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee948cfa1f0c4c65a69fc5204872db0c, user_id=aabb9fdd0abd4b0e9c39fa9e7814c733, role_id=f4e1d6f34f674f539ea7f2564646c0a5)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:20.932683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd374c9-8cc0-4ef5-8254-1b3ab93e0420 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:20.959642 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 68/136] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:00:20 2022] PUT /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c/users/aabb9fdd0abd4b0e9c39fa9e7814c733/roles/f4e1d6f34f674f539ea7f2564646c0a5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:00:21.001123 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdeab8-14fb-4198-8bf1-77b7b0ae028d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:21.005828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:21.047958 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a880ba4-41c5-4568-b967-1fc139a69586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:21.048878 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a880ba4-41c5-4568-b967-1fc139a69586 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:21.049243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a880ba4-41c5-4568-b967-1fc139a69586 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:21.049521 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a880ba4-41c5-4568-b967-1fc139a69586 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c/users/e054eebdda2a494f804b604ef19ea604/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:21.051725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a880ba4-41c5-4568-b967-1fc139a69586 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee948cfa1f0c4c65a69fc5204872db0c, user_id=e054eebdda2a494f804b604ef19ea604, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:21.053218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a880ba4-41c5-4568-b967-1fc139a69586 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:21.080978 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 69/137] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:00:20 2022] PUT /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c/users/e054eebdda2a494f804b604ef19ea604/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:00:21.344366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabc3a3-8e9f-48d5-9c8c-2ea9fbcfe790 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:21.344554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabc3a3-8e9f-48d5-9c8c-2ea9fbcfe790 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:21.344835 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcabc3a3-8e9f-48d5-9c8c-2ea9fbcfe790 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:21.346242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 69/138] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:21.351898 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8cf5b-cd1a-4f78-aebe-d4f2f033e1cf None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:21.352194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8cf5b-cd1a-4f78-aebe-d4f2f033e1cf None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:21.352474 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd8cf5b-cd1a-4f78-aebe-d4f2f033e1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:21.371517 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4dd8cf5b-cd1a-4f78-aebe-d4f2f033e1cf None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:21.372714 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4dd8cf5b-cd1a-4f78-aebe-d4f2f033e1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:21.445118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae099eb-1bfd-442f-8a62-eaa8f95aff74 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:21.445410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae099eb-1bfd-442f-8a62-eaa8f95aff74 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:21.445670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae099eb-1bfd-442f-8a62-eaa8f95aff74 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:21.447036 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 70/139] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:21.451866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd828389-fc42-45fc-953b-2b3a562dbc7a None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:21.452149 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd828389-fc42-45fc-953b-2b3a562dbc7a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:21.452434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd828389-fc42-45fc-953b-2b3a562dbc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:21.455433 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 70/140] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:21.472115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-fd828389-fc42-45fc-953b-2b3a562dbc7a None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:21.473329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fd828389-fc42-45fc-953b-2b3a562dbc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:21.494226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8cf5b-cd1a-4f78-aebe-d4f2f033e1cf None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:21.522621 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:21.545313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 71/141] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:21.569315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd828389-fc42-45fc-953b-2b3a562dbc7a None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:21.649060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:21.661593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd8cf5b-cd1a-4f78-aebe-d4f2f033e1cf None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:21.739269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88613c-4719-4466-8b73-4ff60dbbef75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:21.740224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88613c-4719-4466-8b73-4ff60dbbef75 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:21.740510 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88613c-4719-4466-8b73-4ff60dbbef75 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:21.740794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab88613c-4719-4466-8b73-4ff60dbbef75 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:21.755498 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab88613c-4719-4466-8b73-4ff60dbbef75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-ab88613c-4719-4466-8b73-4ff60dbbef75 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:21.763820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:21.764721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:21.765701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:21.765701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:21.765701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:21.765701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:21.765701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:21.765701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:21.765701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:21.765701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:21.765701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:21.765701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:21.765701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:21.765701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:21.765701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 71/142] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:00:21.772860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88613c-4719-4466-8b73-4ff60dbbef75 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:21.795136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd828389-fc42-45fc-953b-2b3a562dbc7a None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:21.887758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966b629-be6e-4f8f-8a58-91c87cba8668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:21.888991 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966b629-be6e-4f8f-8a58-91c87cba8668 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:21.889498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966b629-be6e-4f8f-8a58-91c87cba8668 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:21.890075 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966b629-be6e-4f8f-8a58-91c87cba8668 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:21.897690 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966b629-be6e-4f8f-8a58-91c87cba8668 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:21.899463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966b629-be6e-4f8f-8a58-91c87cba8668 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:21.901385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 72/143] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:00:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:21.913880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966b629-be6e-4f8f-8a58-91c87cba8668 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:21.925877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab88613c-4719-4466-8b73-4ff60dbbef75 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:21.960516 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111f442-3d2d-4633-a510-c30d30a30eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:21.961222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111f442-3d2d-4633-a510-c30d30a30eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:21.961516 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111f442-3d2d-4633-a510-c30d30a30eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:21.961790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b111f442-3d2d-4633-a510-c30d30a30eb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:21.962539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111f442-3d2d-4633-a510-c30d30a30eb9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:21.962844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111f442-3d2d-4633-a510-c30d30a30eb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:21.964238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b111f442-3d2d-4633-a510-c30d30a30eb9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:21.973923 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 72/144] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:21.979457 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111f442-3d2d-4633-a510-c30d30a30eb9 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:22.002354 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:22.042756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:22.140879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966b629-be6e-4f8f-8a58-91c87cba8668 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:22.156515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b111f442-3d2d-4633-a510-c30d30a30eb9 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:22.202855 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa3d7e-999d-4597-b80b-7e5f8190b98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:22.203553 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa3d7e-999d-4597-b80b-7e5f8190b98c None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:22.203837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa3d7e-999d-4597-b80b-7e5f8190b98c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:22.204118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fa3d7e-999d-4597-b80b-7e5f8190b98c None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:22.205193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa3d7e-999d-4597-b80b-7e5f8190b98c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:22.206894 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fa3d7e-999d-4597-b80b-7e5f8190b98c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:22.268026 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222c015-cbdc-47a9-ab1c-60df11238abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:22.268914 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222c015-cbdc-47a9-ab1c-60df11238abe None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:22.269229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222c015-cbdc-47a9-ab1c-60df11238abe None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:22.269506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c222c015-cbdc-47a9-ab1c-60df11238abe None admin] PATH_INFO: `/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:22.270745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c222c015-cbdc-47a9-ab1c-60df11238abe None admin] RBAC: Authorizing `identity:get_user(user_id=aabb9fdd0abd4b0e9c39fa9e7814c733)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:22.272207 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c222c015-cbdc-47a9-ab1c-60df11238abe None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:22.274072 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 73/145] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:21 2022] GET /identity/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733 => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:22.278145 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 73/146] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:00:21 2022] POST /identity/v3/users => generated 283 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:00:22.281948 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222c015-cbdc-47a9-ab1c-60df11238abe None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:22.420856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:22.443352 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c222c015-cbdc-47a9-ab1c-60df11238abe None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:22.499183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6fc3a-9925-491f-98a1-c3e048c20737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:22.499851 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6fc3a-9925-491f-98a1-c3e048c20737 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:22.500139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6fc3a-9925-491f-98a1-c3e048c20737 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:22.500419 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c6fc3a-9925-491f-98a1-c3e048c20737 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:22.599728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6fc3a-9925-491f-98a1-c3e048c20737 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:22.601248 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c6fc3a-9925-491f-98a1-c3e048c20737 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:22.723994 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:22.742982 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 74/147] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:22.748663 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa3d7e-999d-4597-b80b-7e5f8190b98c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:22.870314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fa3d7e-999d-4597-b80b-7e5f8190b98c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:22.878206 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:22.919383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d963382d-e4fd-4e62-af89-71c0f318c938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:22.920261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963382d-e4fd-4e62-af89-71c0f318c938 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:22.920670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963382d-e4fd-4e62-af89-71c0f318c938 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:22.920967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d963382d-e4fd-4e62-af89-71c0f318c938 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:22.925659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963382d-e4fd-4e62-af89-71c0f318c938 None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:22.927136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d963382d-e4fd-4e62-af89-71c0f318c938 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:22.928876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 74/148] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:22 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:22.935042 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6fc3a-9925-491f-98a1-c3e048c20737 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:23.047075 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ebc02-b51f-4fe9-bdbc-712076ea6d60 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:23.047371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ebc02-b51f-4fe9-bdbc-712076ea6d60 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:23.047568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ebc02-b51f-4fe9-bdbc-712076ea6d60 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:23.049133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 75/149] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:23.054374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47def5-d8f3-4b14-87b6-545a770a1815 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:23.054671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47def5-d8f3-4b14-87b6-545a770a1815 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:23.054936 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47def5-d8f3-4b14-87b6-545a770a1815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:23.081230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c6fc3a-9925-491f-98a1-c3e048c20737 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:23.093939 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4c47def5-d8f3-4b14-87b6-545a770a1815 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:23.095504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4c47def5-d8f3-4b14-87b6-545a770a1815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:23.125662 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc6f24-59f1-4e6c-a435-f794618e6c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:23.126710 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc6f24-59f1-4e6c-a435-f794618e6c3f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:23.127206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc6f24-59f1-4e6c-a435-f794618e6c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:23.127680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc6f24-59f1-4e6c-a435-f794618e6c3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:23.129792 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc6f24-59f1-4e6c-a435-f794618e6c3f None admin] RBAC: Adding query filter params (role.id=3d9688bbbc15422cbfc59ded082fbe41, scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c, user.id=aabb9fdd0abd4b0e9c39fa9e7814c733) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:23.130764 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc6f24-59f1-4e6c-a435-f794618e6c3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:23.132470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdc6f24-59f1-4e6c-a435-f794618e6c3f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:23.141295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 75/150] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:22 2022] GET /identity/v3/role_assignments?user.id=aabb9fdd0abd4b0e9c39fa9e7814c733&scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c&role.id=3d9688bbbc15422cbfc59ded082fbe41 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:23.209648 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 76/151] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:23.246911 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc6f24-59f1-4e6c-a435-f794618e6c3f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:23.325372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:23.387413 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdc6f24-59f1-4e6c-a435-f794618e6c3f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:23.434710 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea71d4-f26a-4b17-b615-e1d95a77b8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:23.435898 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea71d4-f26a-4b17-b615-e1d95a77b8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:23.436427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea71d4-f26a-4b17-b615-e1d95a77b8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:23.436898 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea71d4-f26a-4b17-b615-e1d95a77b8f9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:23.449849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea71d4-f26a-4b17-b615-e1d95a77b8f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-bcea71d4-f26a-4b17-b615-e1d95a77b8f9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:23.458720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:23.459610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:23.460385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:23.460385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:23.460385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:23.460385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:23.460385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:23.460385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:23.460385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:23.460385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:23.460385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:23.460385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:23.460385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:23.460385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:23.462899 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 76/152] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:23.470045 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47def5-d8f3-4b14-87b6-545a770a1815 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:23.616896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47def5-d8f3-4b14-87b6-545a770a1815 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:23.618549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:23.663580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881910e9-c219-4803-a1cb-9dda6d469d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:23.664284 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881910e9-c219-4803-a1cb-9dda6d469d1b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:23.664566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881910e9-c219-4803-a1cb-9dda6d469d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:23.664847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881910e9-c219-4803-a1cb-9dda6d469d1b None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:23.665651 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881910e9-c219-4803-a1cb-9dda6d469d1b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:23.665961 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881910e9-c219-4803-a1cb-9dda6d469d1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:23.667371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881910e9-c219-4803-a1cb-9dda6d469d1b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:23.681026 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 77/153] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:23.686990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea71d4-f26a-4b17-b615-e1d95a77b8f9 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:23.708419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c5d5e-5536-4c2f-91d1-fd2f9516a6ba None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:23.708729 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c5d5e-5536-4c2f-91d1-fd2f9516a6ba None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:23.709005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c5d5e-5536-4c2f-91d1-fd2f9516a6ba None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:23.710592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 78/154] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:00:23.716012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b35b4-e7aa-4d09-b78e-6de74123e782 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:23.716302 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b35b4-e7aa-4d09-b78e-6de74123e782 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:23.716582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b35b4-e7aa-4d09-b78e-6de74123e782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:23.740454 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-da9b35b4-e7aa-4d09-b78e-6de74123e782 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:23.741690 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-da9b35b4-e7aa-4d09-b78e-6de74123e782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:23.821652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea71d4-f26a-4b17-b615-e1d95a77b8f9 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:23.855999 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:23.859064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358845e2-ad91-4f04-8ada-2cd1ee920521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:23.859965 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358845e2-ad91-4f04-8ada-2cd1ee920521 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:23.860233 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358845e2-ad91-4f04-8ada-2cd1ee920521 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:23.860503 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358845e2-ad91-4f04-8ada-2cd1ee920521 None admin] PATH_INFO: `/v3/users/57c664aabb3c487e88c8884d7ede941f` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:23.861763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358845e2-ad91-4f04-8ada-2cd1ee920521 None admin] RBAC: Authorizing `identity:get_user(user_id=57c664aabb3c487e88c8884d7ede941f)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:23.863236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358845e2-ad91-4f04-8ada-2cd1ee920521 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:23.865208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 77/155] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:23 2022] GET /identity/v3/users/57c664aabb3c487e88c8884d7ede941f => generated 289 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:23.870935 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358845e2-ad91-4f04-8ada-2cd1ee920521 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:23.889653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 79/156] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:23.912670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b35b4-e7aa-4d09-b78e-6de74123e782 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:24.014369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:24.021050 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b35b4-e7aa-4d09-b78e-6de74123e782 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:24.040960 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358845e2-ad91-4f04-8ada-2cd1ee920521 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:24.056211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f527fb3-6c68-4c91-a0a1-ae60b3b09ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:24.057114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f527fb3-6c68-4c91-a0a1-ae60b3b09ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:24.057399 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f527fb3-6c68-4c91-a0a1-ae60b3b09ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:24.057677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f527fb3-6c68-4c91-a0a1-ae60b3b09ed1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:24.069315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f527fb3-6c68-4c91-a0a1-ae60b3b09ed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-1f527fb3-6c68-4c91-a0a1-ae60b3b09ed1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:24.078119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:24.079194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:24.080326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:24.080326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:24.080326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:24.080326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:24.080326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:24.080326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:24.080326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:24.080326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:24.080326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:24.080326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:24.080326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:24.080326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:24.080326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 80/157] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:24.083512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1280a-7ead-41a0-90f4-84a47f479248 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:24.083512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1280a-7ead-41a0-90f4-84a47f479248 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:24.083512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c1280a-7ead-41a0-90f4-84a47f479248 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:24.083512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 81/158] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:24.086942 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1280a-7ead-41a0-90f4-84a47f479248 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:24.106663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4428ff-8439-4347-ae82-d24f7d96d4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:24.108098 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4428ff-8439-4347-ae82-d24f7d96d4fe None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:24.108383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4428ff-8439-4347-ae82-d24f7d96d4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:24.108655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4428ff-8439-4347-ae82-d24f7d96d4fe None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:24.111856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4428ff-8439-4347-ae82-d24f7d96d4fe None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:24.113379 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4428ff-8439-4347-ae82-d24f7d96d4fe None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:24.115138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 78/159] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:23 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:24.117387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac7357-80e9-48f1-8cbc-8a47feee505d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:24.117677 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac7357-80e9-48f1-8cbc-8a47feee505d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:24.117959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac7357-80e9-48f1-8cbc-8a47feee505d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:24.150009 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2dac7357-80e9-48f1-8cbc-8a47feee505d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:24.151223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2dac7357-80e9-48f1-8cbc-8a47feee505d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:24.197020 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c1280a-7ead-41a0-90f4-84a47f479248 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:24.219577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:24.234662 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6cacf-c577-401f-89e4-824d533c6211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:24.235604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6cacf-c577-401f-89e4-824d533c6211 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:24.236048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6cacf-c577-401f-89e4-824d533c6211 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:24.236466 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d6cacf-c577-401f-89e4-824d533c6211 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:24.237422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6cacf-c577-401f-89e4-824d533c6211 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:24.237865 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6cacf-c577-401f-89e4-824d533c6211 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:24.239492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d6cacf-c577-401f-89e4-824d533c6211 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:24.250820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 82/160] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:24.251930 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6cacf-c577-401f-89e4-824d533c6211 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:24.285359 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:24.286313 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 79/161] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:24.287289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac7357-80e9-48f1-8cbc-8a47feee505d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:24.400643 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac7357-80e9-48f1-8cbc-8a47feee505d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:24.425398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d6cacf-c577-401f-89e4-824d533c6211 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:24.486187 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccfe58-6b34-455e-83a3-a84493c2fd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:24.487093 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccfe58-6b34-455e-83a3-a84493c2fd31 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:24.487375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccfe58-6b34-455e-83a3-a84493c2fd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:24.487658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ccfe58-6b34-455e-83a3-a84493c2fd31 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c/users/57c664aabb3c487e88c8884d7ede941f/roles/73fb34f5dbe1404fa7ec015422951829` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:24.489451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b3bc0-5f67-4bc8-aeba-ab4609404227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:24.490538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b3bc0-5f67-4bc8-aeba-ab4609404227 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:24.491000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b3bc0-5f67-4bc8-aeba-ab4609404227 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:24.491466 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79b3bc0-5f67-4bc8-aeba-ab4609404227 None admin] PATH_INFO: `/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:24.492836 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b3bc0-5f67-4bc8-aeba-ab4609404227 None admin] RBAC: Authorizing `identity:get_user(user_id=aabb9fdd0abd4b0e9c39fa9e7814c733)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:24.494248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccfe58-6b34-455e-83a3-a84493c2fd31 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee948cfa1f0c4c65a69fc5204872db0c, user_id=57c664aabb3c487e88c8884d7ede941f, role_id=73fb34f5dbe1404fa7ec015422951829)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:24.494925 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79b3bc0-5f67-4bc8-aeba-ab4609404227 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:24.495810 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ccfe58-6b34-455e-83a3-a84493c2fd31 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:24.501025 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 80/162] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:24 2022] GET /identity/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733 => generated 275 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:24.502086 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b3bc0-5f67-4bc8-aeba-ab4609404227 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:24.520151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 83/163] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:00:24 2022] PUT /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c/users/57c664aabb3c487e88c8884d7ede941f/roles/73fb34f5dbe1404fa7ec015422951829 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:00:24.521024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccfe58-6b34-455e-83a3-a84493c2fd31 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:24.576610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:24.702822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79b3bc0-5f67-4bc8-aeba-ab4609404227 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:24.705326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:24.742329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ccfe58-6b34-455e-83a3-a84493c2fd31 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:24.774869 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d13584-4ac3-47f4-acc7-961dc414d905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:24.775686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d13584-4ac3-47f4-acc7-961dc414d905 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:24.775978 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d13584-4ac3-47f4-acc7-961dc414d905 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:24.776349 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d13584-4ac3-47f4-acc7-961dc414d905 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:24.788406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85de6d-fd8e-4c6b-880c-7c34accbcb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:24.789090 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85de6d-fd8e-4c6b-880c-7c34accbcb1b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:24.789369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85de6d-fd8e-4c6b-880c-7c34accbcb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:24.789640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85de6d-fd8e-4c6b-880c-7c34accbcb1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:24.809934 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d13584-4ac3-47f4-acc7-961dc414d905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-25d13584-4ac3-47f4-acc7-961dc414d905 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:24.838226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:24.839605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:24.840702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:24.840702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:24.840702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:24.840702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:24.840702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:24.840702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:24.840702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:24.840702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:24.840702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:24.840702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:24.840702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:24.840702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:24.840702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 81/164] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:00:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:24.843324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a614ad9-24da-4af0-987f-7b48518cc91f None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:24.843324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a614ad9-24da-4af0-987f-7b48518cc91f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:24.843324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a614ad9-24da-4af0-987f-7b48518cc91f None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:24.843324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 82/165] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:24.848186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb050be-9d65-43ae-be38-0eb96aae1d4b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:24.848469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb050be-9d65-43ae-be38-0eb96aae1d4b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:24.848727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb050be-9d65-43ae-be38-0eb96aae1d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:24.867993 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-dcb050be-9d65-43ae-be38-0eb96aae1d4b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:24.870037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-dcb050be-9d65-43ae-be38-0eb96aae1d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:24.880740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85de6d-fd8e-4c6b-880c-7c34accbcb1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:24.890380 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85de6d-fd8e-4c6b-880c-7c34accbcb1b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:24.959012 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 83/166] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:24.960127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb050be-9d65-43ae-be38-0eb96aae1d4b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:24.994815 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:25.006720 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:25.057609 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 84/167] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:25.058790 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85de6d-fd8e-4c6b-880c-7c34accbcb1b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:25.080642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb050be-9d65-43ae-be38-0eb96aae1d4b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:25.119250 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651b75a-333e-4555-b957-df7ed12f8c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:25.119928 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651b75a-333e-4555-b957-df7ed12f8c57 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:25.120214 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651b75a-333e-4555-b957-df7ed12f8c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:25.121267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4651b75a-333e-4555-b957-df7ed12f8c57 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:25.122040 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651b75a-333e-4555-b957-df7ed12f8c57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:25.122440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651b75a-333e-4555-b957-df7ed12f8c57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:25.126641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4651b75a-333e-4555-b957-df7ed12f8c57 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:25.147598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 84/168] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:25.148497 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651b75a-333e-4555-b957-df7ed12f8c57 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:25.277223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:25.286566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4651b75a-333e-4555-b957-df7ed12f8c57 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:25.308939 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85de6d-fd8e-4c6b-880c-7c34accbcb1b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:25.345307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb7dbb-e84a-4100-84ac-081631ce9218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:25.346179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb7dbb-e84a-4100-84ac-081631ce9218 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:25.346461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb7dbb-e84a-4100-84ac-081631ce9218 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:25.346738 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbb7dbb-e84a-4100-84ac-081631ce9218 None admin] PATH_INFO: `/v3/projects/2f57e9f519db4d1c9d4e343df51aa79e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:25.352570 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb7dbb-e84a-4100-84ac-081631ce9218 None admin] RBAC: Authorizing `identity:get_project(project_id=2f57e9f519db4d1c9d4e343df51aa79e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:25.354126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbb7dbb-e84a-4100-84ac-081631ce9218 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:25.355901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 85/169] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:25 2022] GET /identity/v3/projects/2f57e9f519db4d1c9d4e343df51aa79e => generated 312 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:25.356668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb7dbb-e84a-4100-84ac-081631ce9218 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:25.397152 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f5a68-5aa2-4480-b77f-354f5d132631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:25.398209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f5a68-5aa2-4480-b77f-354f5d132631 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:25.398654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f5a68-5aa2-4480-b77f-354f5d132631 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:25.399103 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f5a68-5aa2-4480-b77f-354f5d132631 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:25.426861 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f5a68-5aa2-4480-b77f-354f5d132631 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-b58f5a68-5aa2-4480-b77f-354f5d132631 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:25.444549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:25.445766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:25.446753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:25.446753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:25.446753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:25.446753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:25.446753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:25.446753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:25.446753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:25.446753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:25.446753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:25.446753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:25.446753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:25.446753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:25.449918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 85/170] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:00:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 391 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:25.451402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f5a68-5aa2-4480-b77f-354f5d132631 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:25.519790 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:25.598090 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f5a68-5aa2-4480-b77f-354f5d132631 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:25.624451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbb7dbb-e84a-4100-84ac-081631ce9218 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:25.632881 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e949149-3ac8-4881-add3-61eb028abead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:25.633577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e949149-3ac8-4881-add3-61eb028abead None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:25.633862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e949149-3ac8-4881-add3-61eb028abead None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:25.634137 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e949149-3ac8-4881-add3-61eb028abead None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:25.638386 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e949149-3ac8-4881-add3-61eb028abead None admin] RBAC: Adding query filter params (role.id=3d9688bbbc15422cbfc59ded082fbe41, scope.project.id=2f57e9f519db4d1c9d4e343df51aa79e, user.id=aabb9fdd0abd4b0e9c39fa9e7814c733) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:25.638386 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e949149-3ac8-4881-add3-61eb028abead None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:25.638386 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e949149-3ac8-4881-add3-61eb028abead None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:25.644299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 86/171] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:25 2022] GET /identity/v3/role_assignments?user.id=aabb9fdd0abd4b0e9c39fa9e7814c733&scope.project.id=2f57e9f519db4d1c9d4e343df51aa79e&role.id=3d9688bbbc15422cbfc59ded082fbe41 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:25.645216 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e949149-3ac8-4881-add3-61eb028abead None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:25.714221 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b25911-07a4-4597-b8bc-fa18a56106c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:25.715355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b25911-07a4-4597-b8bc-fa18a56106c0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:25.715902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b25911-07a4-4597-b8bc-fa18a56106c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:25.716900 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b25911-07a4-4597-b8bc-fa18a56106c0 None admin] PATH_INFO: `/v3/users/e054eebdda2a494f804b604ef19ea604` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:25.718290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b25911-07a4-4597-b8bc-fa18a56106c0 None admin] RBAC: Authorizing `identity:get_user(user_id=e054eebdda2a494f804b604ef19ea604)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:25.720003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b25911-07a4-4597-b8bc-fa18a56106c0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:25.728628 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 86/172] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:25 2022] GET /identity/v3/users/e054eebdda2a494f804b604ef19ea604 => generated 247 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:25.733875 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b25911-07a4-4597-b8bc-fa18a56106c0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:25.752421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e949149-3ac8-4881-add3-61eb028abead None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:25.773462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:25.805716 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751adae8-3968-4855-9ba9-51aca23d87e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:25.806429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751adae8-3968-4855-9ba9-51aca23d87e8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:25.806711 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751adae8-3968-4855-9ba9-51aca23d87e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:25.806993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751adae8-3968-4855-9ba9-51aca23d87e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:25.807768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751adae8-3968-4855-9ba9-51aca23d87e8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:25.808080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751adae8-3968-4855-9ba9-51aca23d87e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:25.809522 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751adae8-3968-4855-9ba9-51aca23d87e8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:25.811112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:25.823264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 87/173] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:00:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:25.831021 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751adae8-3968-4855-9ba9-51aca23d87e8 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:25.961178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b25911-07a4-4597-b8bc-fa18a56106c0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:25.993919 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751adae8-3968-4855-9ba9-51aca23d87e8 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:26.052613 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4994e-488e-4034-bfef-71de72fd1c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:26.053456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4994e-488e-4034-bfef-71de72fd1c21 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:26.053736 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4994e-488e-4034-bfef-71de72fd1c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:26.054009 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4994e-488e-4034-bfef-71de72fd1c21 None admin] PATH_INFO: `/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:26.056116 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe89c80-2996-455c-a60a-6c604d72553d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:26.056773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe89c80-2996-455c-a60a-6c604d72553d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:26.057080 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe89c80-2996-455c-a60a-6c604d72553d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:26.057356 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe89c80-2996-455c-a60a-6c604d72553d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:26.058565 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4994e-488e-4034-bfef-71de72fd1c21 None admin] RBAC: Authorizing `identity:get_user(user_id=aabb9fdd0abd4b0e9c39fa9e7814c733)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:26.062877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee4994e-488e-4034-bfef-71de72fd1c21 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:26.065614 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 88/174] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:25 2022] GET /identity/v3/users/aabb9fdd0abd4b0e9c39fa9e7814c733 => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:26.070974 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4994e-488e-4034-bfef-71de72fd1c21 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:26.141281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:26.217861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe89c80-2996-455c-a60a-6c604d72553d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:26.219347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe89c80-2996-455c-a60a-6c604d72553d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:26.295471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4994e-488e-4034-bfef-71de72fd1c21 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:26.299573 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:26.360572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f89265-2dd7-43e0-894f-17488102eff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:26.361823 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f89265-2dd7-43e0-894f-17488102eff6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:26.361823 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f89265-2dd7-43e0-894f-17488102eff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:26.361823 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f89265-2dd7-43e0-894f-17488102eff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:26.437874 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f89265-2dd7-43e0-894f-17488102eff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:26.439366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f89265-2dd7-43e0-894f-17488102eff6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:26.475240 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:26.478445 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 87/175] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 745 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:00:26.484422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9ab37-b98b-4198-9c95-dab35bc9a63f None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:26.484716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9ab37-b98b-4198-9c95-dab35bc9a63f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:26.485020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9ab37-b98b-4198-9c95-dab35bc9a63f None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:26.486467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 88/176] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:26.491098 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9ab37-b98b-4198-9c95-dab35bc9a63f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:26.553506 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 89/177] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:00:26.554742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0fa33-3e94-44d3-b3a8-810ff5b5184b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:26.555037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0fa33-3e94-44d3-b3a8-810ff5b5184b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:26.555397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a0fa33-3e94-44d3-b3a8-810ff5b5184b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:26.581931 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-22a0fa33-3e94-44d3-b3a8-810ff5b5184b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:26.583129 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-22a0fa33-3e94-44d3-b3a8-810ff5b5184b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:26.683028 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9ab37-b98b-4198-9c95-dab35bc9a63f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:26.703691 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 90/178] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:26.704637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0fa33-3e94-44d3-b3a8-810ff5b5184b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:26.723860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:26.735953 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15554b82-835a-41ee-ab5f-a8571ac08e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:26.736959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15554b82-835a-41ee-ab5f-a8571ac08e09 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:26.737227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15554b82-835a-41ee-ab5f-a8571ac08e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:26.737499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15554b82-835a-41ee-ab5f-a8571ac08e09 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:26.741751 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15554b82-835a-41ee-ab5f-a8571ac08e09 None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:26.743276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15554b82-835a-41ee-ab5f-a8571ac08e09 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:26.745153 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 89/179] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:26 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:26.745897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15554b82-835a-41ee-ab5f-a8571ac08e09 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:26.765141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:26.807749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a0fa33-3e94-44d3-b3a8-810ff5b5184b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:26.843947 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab3a24-faba-4077-88f8-d27ef9509a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:26.844911 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab3a24-faba-4077-88f8-d27ef9509a20 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:26.845292 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab3a24-faba-4077-88f8-d27ef9509a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:26.845593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab3a24-faba-4077-88f8-d27ef9509a20 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:26.846775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab3a24-faba-4077-88f8-d27ef9509a20 None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:26.848224 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab3a24-faba-4077-88f8-d27ef9509a20 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:26.850138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 91/180] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:26 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:26.855452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab3a24-faba-4077-88f8-d27ef9509a20 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:26.885937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15554b82-835a-41ee-ab5f-a8571ac08e09 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:26.923392 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca4bb6-068a-4295-bbf0-ded852f317c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:26.924645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca4bb6-068a-4295-bbf0-ded852f317c5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:26.925204 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca4bb6-068a-4295-bbf0-ded852f317c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:26.925822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca4bb6-068a-4295-bbf0-ded852f317c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:26.937127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca4bb6-068a-4295-bbf0-ded852f317c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-0fca4bb6-068a-4295-bbf0-ded852f317c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:26.945543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:26.946421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:26.947197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:26.947197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:26.947197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:26.947197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:26.947197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:26.947197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:26.947197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:26.947197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:26.947197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:26.947197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:26.947197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:26.947197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:26.948253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 90/181] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:00:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:26.952997 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f518c4-cb15-45cd-8e13-9b980769eeb3 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:26.952997 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f518c4-cb15-45cd-8e13-9b980769eeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:26.952997 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f518c4-cb15-45cd-8e13-9b980769eeb3 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:26.952997 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 91/182] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:26.954957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f518c4-cb15-45cd-8e13-9b980769eeb3 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:26.959942 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab3a24-faba-4077-88f8-d27ef9509a20 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:26.988456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:26.997934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eed4c7-7d7a-4ded-a578-0a9515017a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:26.998660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eed4c7-7d7a-4ded-a578-0a9515017a41 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:26.998893 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eed4c7-7d7a-4ded-a578-0a9515017a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:26.999172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eed4c7-7d7a-4ded-a578-0a9515017a41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:27.000389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eed4c7-7d7a-4ded-a578-0a9515017a41 None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c, user.id=e054eebdda2a494f804b604ef19ea604) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:27.001051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eed4c7-7d7a-4ded-a578-0a9515017a41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:27.002805 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eed4c7-7d7a-4ded-a578-0a9515017a41 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:27.016490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 92/183] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:26 2022] GET /identity/v3/role_assignments?user.id=e054eebdda2a494f804b604ef19ea604&scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:27.016490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c39e5-6851-46df-bb9a-a0819f2e1a9f None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:27.016490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c39e5-6851-46df-bb9a-a0819f2e1a9f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:27.016490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c39e5-6851-46df-bb9a-a0819f2e1a9f None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:27.016490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 93/184] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:27.016490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c39e5-6851-46df-bb9a-a0819f2e1a9f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:27.021266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:27.071155 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f518c4-cb15-45cd-8e13-9b980769eeb3 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:27.111901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6386283-1a48-47ef-a986-844638bb3374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:27.112579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6386283-1a48-47ef-a986-844638bb3374 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:27.112865 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6386283-1a48-47ef-a986-844638bb3374 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:27.113174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6386283-1a48-47ef-a986-844638bb3374 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:27.114049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6386283-1a48-47ef-a986-844638bb3374 None admin] RBAC: Adding query filter params (role.id=f4e1d6f34f674f539ea7f2564646c0a5, scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c, user.id=aabb9fdd0abd4b0e9c39fa9e7814c733) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:27.114367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6386283-1a48-47ef-a986-844638bb3374 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:27.115858 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6386283-1a48-47ef-a986-844638bb3374 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:27.122648 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 92/185] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:26 2022] GET /identity/v3/role_assignments?user.id=aabb9fdd0abd4b0e9c39fa9e7814c733&scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c&role.id=f4e1d6f34f674f539ea7f2564646c0a5 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:27.124100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ba124-d52c-485a-ada7-107dbe6d752a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:27.124395 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ba124-d52c-485a-ada7-107dbe6d752a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:27.124677 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ba124-d52c-485a-ada7-107dbe6d752a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:27.143903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-394ba124-d52c-485a-ada7-107dbe6d752a None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:27.146259 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c39e5-6851-46df-bb9a-a0819f2e1a9f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:27.153883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-394ba124-d52c-485a-ada7-107dbe6d752a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:27.192586 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cf7835-ad93-49c7-b9ec-96eb94372274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:27.193617 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf7835-ad93-49c7-b9ec-96eb94372274 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:27.194735 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf7835-ad93-49c7-b9ec-96eb94372274 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:27.195230 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cf7835-ad93-49c7-b9ec-96eb94372274 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:27.196299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf7835-ad93-49c7-b9ec-96eb94372274 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:27.196794 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf7835-ad93-49c7-b9ec-96eb94372274 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:27.199200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cf7835-ad93-49c7-b9ec-96eb94372274 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:27.211786 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 94/186] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:27.213807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44fe06-50e7-489b-b6cb-cff496966fa7 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:27.214319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44fe06-50e7-489b-b6cb-cff496966fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:27.215963 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44fe06-50e7-489b-b6cb-cff496966fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:27.236968 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ea44fe06-50e7-489b-b6cb-cff496966fa7 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:27.238543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ea44fe06-50e7-489b-b6cb-cff496966fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:27.276458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 93/187] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:27.280676 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ba124-d52c-485a-ada7-107dbe6d752a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:27.329639 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:27.351587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 95/188] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:27.353433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44fe06-50e7-489b-b6cb-cff496966fa7 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:27.364061 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:27.427782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ba124-d52c-485a-ada7-107dbe6d752a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:27.477680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ced8c-77d9-44b6-91ee-98c019e6960c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:27.478480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ced8c-77d9-44b6-91ee-98c019e6960c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:27.478752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ced8c-77d9-44b6-91ee-98c019e6960c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:27.479014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ced8c-77d9-44b6-91ee-98c019e6960c None admin] PATH_INFO: `/v3/users/c255f73da79249a2b651aa65b5184f59` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:27.486249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44fe06-50e7-489b-b6cb-cff496966fa7 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:27.495799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ced8c-77d9-44b6-91ee-98c019e6960c None admin] RBAC: Authorizing `identity:get_user(user_id=c255f73da79249a2b651aa65b5184f59)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:27.499781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ced8c-77d9-44b6-91ee-98c019e6960c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:27.504979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 94/189] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:27 2022] GET /identity/v3/users/c255f73da79249a2b651aa65b5184f59 => generated 283 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:27.506133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ced8c-77d9-44b6-91ee-98c019e6960c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:27.530961 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4233c-35a1-4c98-9d06-fa85e356fcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:27.531764 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4233c-35a1-4c98-9d06-fa85e356fcd3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:27.532047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4233c-35a1-4c98-9d06-fa85e356fcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:27.532318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d4233c-35a1-4c98-9d06-fa85e356fcd3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:27.533820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4233c-35a1-4c98-9d06-fa85e356fcd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:27.534968 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d4233c-35a1-4c98-9d06-fa85e356fcd3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:27.536737 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 96/190] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:00:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:27.542311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4233c-35a1-4c98-9d06-fa85e356fcd3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:27.593710 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:27.607780 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ced8c-77d9-44b6-91ee-98c019e6960c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:27.618900 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:27.643674 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dd154-0b34-42ce-846b-7491184d84df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:27.644477 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dd154-0b34-42ce-846b-7491184d84df None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:27.644755 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dd154-0b34-42ce-846b-7491184d84df None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:27.644866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d4233c-35a1-4c98-9d06-fa85e356fcd3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:27.645064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dd154-0b34-42ce-846b-7491184d84df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:27.655371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293dd154-0b34-42ce-846b-7491184d84df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-293dd154-0b34-42ce-846b-7491184d84df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:27.663409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:27.664461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:27.665316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:27.665316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:27.665316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:27.665316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:27.665316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:27.665316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:27.665316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:27.665316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:27.665316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:27.665316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:27.665316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:27.665316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:27.665316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 95/191] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:00:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:27.665965 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dd154-0b34-42ce-846b-7491184d84df None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:27.684456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8855c9-3992-4a77-b87d-e5481192232d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:27.685138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8855c9-3992-4a77-b87d-e5481192232d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:27.685413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8855c9-3992-4a77-b87d-e5481192232d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:27.685678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8855c9-3992-4a77-b87d-e5481192232d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:27.751836 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8855c9-3992-4a77-b87d-e5481192232d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:27.753248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8855c9-3992-4a77-b87d-e5481192232d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:27.765506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dd154-0b34-42ce-846b-7491184d84df None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:27.798893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8d340-69f7-43d9-a731-cce31fd6b94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:27.799556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8d340-69f7-43d9-a731-cce31fd6b94e None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:27.799837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8d340-69f7-43d9-a731-cce31fd6b94e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:27.800138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8d340-69f7-43d9-a731-cce31fd6b94e None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:27.800849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8d340-69f7-43d9-a731-cce31fd6b94e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:27.802738 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e8d340-69f7-43d9-a731-cce31fd6b94e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:27.802953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:27.822696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 96/192] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:00:27 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:00:27.823957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c94e69-af85-4ec9-9e41-53aa8d04c7a0 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:27.824268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c94e69-af85-4ec9-9e41-53aa8d04c7a0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:27.824541 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c94e69-af85-4ec9-9e41-53aa8d04c7a0 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:27.826021 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 97/193] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:27.826560 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c94e69-af85-4ec9-9e41-53aa8d04c7a0 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:27.859482 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 97/194] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:27.860817 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a54346-fed5-454d-a88a-18a92b22b488 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:27.861145 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a54346-fed5-454d-a88a-18a92b22b488 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:27.861423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a54346-fed5-454d-a88a-18a92b22b488 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:27.876555 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:27.880169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-00a54346-fed5-454d-a88a-18a92b22b488 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:27.881459 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-00a54346-fed5-454d-a88a-18a92b22b488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:27.954142 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 98/195] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:27.961977 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a54346-fed5-454d-a88a-18a92b22b488 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:27.966169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c94e69-af85-4ec9-9e41-53aa8d04c7a0 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:28.012624 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb9621-3bf8-4978-a28e-25f8a56084ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:28.012817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb9621-3bf8-4978-a28e-25f8a56084ac None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:28.013068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb9621-3bf8-4978-a28e-25f8a56084ac None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:28.013294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb9621-3bf8-4978-a28e-25f8a56084ac None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:28.014055 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb9621-3bf8-4978-a28e-25f8a56084ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:28.014359 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb9621-3bf8-4978-a28e-25f8a56084ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:28.015753 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb9621-3bf8-4978-a28e-25f8a56084ac None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:28.030762 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 98/196] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:00:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:28.030762 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb9621-3bf8-4978-a28e-25f8a56084ac None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:28.071267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:28.098290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a54346-fed5-454d-a88a-18a92b22b488 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:28.138373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc332c91-8477-460e-8577-71e4b47305ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:28.139247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc332c91-8477-460e-8577-71e4b47305ee None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:28.139535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc332c91-8477-460e-8577-71e4b47305ee None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:28.139816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc332c91-8477-460e-8577-71e4b47305ee None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:28.141110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc332c91-8477-460e-8577-71e4b47305ee None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:28.142575 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc332c91-8477-460e-8577-71e4b47305ee None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:28.144595 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 99/197] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:27 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:28.145435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc332c91-8477-460e-8577-71e4b47305ee None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:28.149497 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:28.165762 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb9621-3bf8-4978-a28e-25f8a56084ac None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:28.202259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6961e-c63e-4437-b357-726cca1aada2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:28.203371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6961e-c63e-4437-b357-726cca1aada2 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:28.203703 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6961e-c63e-4437-b357-726cca1aada2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:28.203985 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6961e-c63e-4437-b357-726cca1aada2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:28.215475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc6961e-c63e-4437-b357-726cca1aada2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-9fc6961e-c63e-4437-b357-726cca1aada2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:28.223396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:28.224303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:28.225163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:28.225163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:28.225163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:28.225163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:28.225163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:28.225163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:28.225163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:28.225163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:28.225163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:28.225163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:28.225163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:28.225163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:28.225163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 99/198] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:28.225814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6961e-c63e-4437-b357-726cca1aada2 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:28.272567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc332c91-8477-460e-8577-71e4b47305ee None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:28.319428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf579db-d4fb-4430-9762-478e6e12d518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:28.320440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf579db-d4fb-4430-9762-478e6e12d518 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:28.320788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf579db-d4fb-4430-9762-478e6e12d518 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:28.321205 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf579db-d4fb-4430-9762-478e6e12d518 None admin] PATH_INFO: `/v3/users/c255f73da79249a2b651aa65b5184f59` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:28.322494 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf579db-d4fb-4430-9762-478e6e12d518 None admin] RBAC: Authorizing `identity:get_user(user_id=c255f73da79249a2b651aa65b5184f59)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:28.324073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf579db-d4fb-4430-9762-478e6e12d518 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:28.327434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 100/199] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:28 2022] GET /identity/v3/users/c255f73da79249a2b651aa65b5184f59 => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:28.327805 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf579db-d4fb-4430-9762-478e6e12d518 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:28.347551 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:28.370152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc6961e-c63e-4437-b357-726cca1aada2 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:28.413067 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7ada6-41eb-495b-8417-be25e23b3a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:28.413829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7ada6-41eb-495b-8417-be25e23b3a64 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:28.414177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7ada6-41eb-495b-8417-be25e23b3a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:28.414518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7ada6-41eb-495b-8417-be25e23b3a64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:28.415419 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7ada6-41eb-495b-8417-be25e23b3a64 None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=00a316c6194d45b392adff3ea7182a1c, user.id=c255f73da79249a2b651aa65b5184f59) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:28.416203 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7ada6-41eb-495b-8417-be25e23b3a64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:28.417416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7ada6-41eb-495b-8417-be25e23b3a64 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:28.424230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 100/200] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:28 2022] GET /identity/v3/role_assignments?user.id=c255f73da79249a2b651aa65b5184f59&scope.project.id=00a316c6194d45b392adff3ea7182a1c&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:28.425268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7ada6-41eb-495b-8417-be25e23b3a64 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:28.442530 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:28.453163 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf579db-d4fb-4430-9762-478e6e12d518 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:28.531922 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c850ee-6536-443e-9b3f-6606223a3de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:28.532533 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c850ee-6536-443e-9b3f-6606223a3de0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:28.532811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c850ee-6536-443e-9b3f-6606223a3de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:28.533109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c850ee-6536-443e-9b3f-6606223a3de0 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:28.533879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c850ee-6536-443e-9b3f-6606223a3de0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:28.534186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c850ee-6536-443e-9b3f-6606223a3de0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:28.535587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c850ee-6536-443e-9b3f-6606223a3de0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:28.558277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 101/201] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:28.563518 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c850ee-6536-443e-9b3f-6606223a3de0 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:28.621937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7ada6-41eb-495b-8417-be25e23b3a64 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:28.656975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1afce6-8a31-4075-9487-70b5acce2bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:28.657906 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1afce6-8a31-4075-9487-70b5acce2bca None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:28.658129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1afce6-8a31-4075-9487-70b5acce2bca None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:28.658390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1afce6-8a31-4075-9487-70b5acce2bca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:28.729284 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:28.737144 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1afce6-8a31-4075-9487-70b5acce2bca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:28.738727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1afce6-8a31-4075-9487-70b5acce2bca None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:28.753641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:28.771329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c850ee-6536-443e-9b3f-6606223a3de0 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:28.843044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 101/202] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:28.845520 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bce82-9444-4add-a4b4-f560766bbc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:28.846460 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bce82-9444-4add-a4b4-f560766bbc86 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:28.846737 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bce82-9444-4add-a4b4-f560766bbc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:28.847057 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10bce82-9444-4add-a4b4-f560766bbc86 None admin] PATH_INFO: `/v3/users/57c664aabb3c487e88c8884d7ede941f` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:28.848419 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1afce6-8a31-4075-9487-70b5acce2bca None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:28.853677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10bce82-9444-4add-a4b4-f560766bbc86 None admin] RBAC: Authorizing `identity:get_user(user_id=57c664aabb3c487e88c8884d7ede941f)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:28.855745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10bce82-9444-4add-a4b4-f560766bbc86 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:28.857430 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 102/203] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:28 2022] GET /identity/v3/users/57c664aabb3c487e88c8884d7ede941f => generated 289 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:28.863993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bce82-9444-4add-a4b4-f560766bbc86 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:28.973877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1afce6-8a31-4075-9487-70b5acce2bca None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:28.989054 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:29.020814 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10bce82-9444-4add-a4b4-f560766bbc86 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:29.053391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66feede2-baaa-409e-b98b-554b045e1003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:29.054410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66feede2-baaa-409e-b98b-554b045e1003 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:29.054650 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66feede2-baaa-409e-b98b-554b045e1003 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:29.055024 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66feede2-baaa-409e-b98b-554b045e1003 None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:29.055584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178f897-f6a2-4255-92c8-8214c1cd5ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:29.056178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178f897-f6a2-4255-92c8-8214c1cd5ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:29.056461 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178f897-f6a2-4255-92c8-8214c1cd5ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:29.056738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178f897-f6a2-4255-92c8-8214c1cd5ba3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:29.061675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66feede2-baaa-409e-b98b-554b045e1003 None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:29.063159 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66feede2-baaa-409e-b98b-554b045e1003 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:29.065252 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 102/204] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:28 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:29.081913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66feede2-baaa-409e-b98b-554b045e1003 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:29.082783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:29.137637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178f897-f6a2-4255-92c8-8214c1cd5ba3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:29.139102 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178f897-f6a2-4255-92c8-8214c1cd5ba3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:29.248512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 103/205] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:29.258259 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178f897-f6a2-4255-92c8-8214c1cd5ba3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:29.287810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66feede2-baaa-409e-b98b-554b045e1003 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:29.323225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31517e-f335-4952-ae3e-b0ccd6d89568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:29.323927 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31517e-f335-4952-ae3e-b0ccd6d89568 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:29.324206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31517e-f335-4952-ae3e-b0ccd6d89568 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:29.324477 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31517e-f335-4952-ae3e-b0ccd6d89568 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:29.325339 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31517e-f335-4952-ae3e-b0ccd6d89568 None admin] RBAC: Adding query filter params (role.id=f4e1d6f34f674f539ea7f2564646c0a5, scope.project.id=00a316c6194d45b392adff3ea7182a1c, user.id=c255f73da79249a2b651aa65b5184f59) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:29.325647 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31517e-f335-4952-ae3e-b0ccd6d89568 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:29.327387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31517e-f335-4952-ae3e-b0ccd6d89568 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:29.333458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 103/206] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:29 2022] GET /identity/v3/role_assignments?user.id=c255f73da79249a2b651aa65b5184f59&scope.project.id=00a316c6194d45b392adff3ea7182a1c&role.id=f4e1d6f34f674f539ea7f2564646c0a5 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:29.365141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:29.431541 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178f897-f6a2-4255-92c8-8214c1cd5ba3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:29.468557 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16773db8-f9ad-4f34-8f4e-55248347e218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:29.469476 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16773db8-f9ad-4f34-8f4e-55248347e218 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:29.469757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16773db8-f9ad-4f34-8f4e-55248347e218 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:29.470030 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16773db8-f9ad-4f34-8f4e-55248347e218 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:29.471317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16773db8-f9ad-4f34-8f4e-55248347e218 None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:29.472778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16773db8-f9ad-4f34-8f4e-55248347e218 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:29.474860 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 104/207] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:29 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:29.480724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31517e-f335-4952-ae3e-b0ccd6d89568 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:29.545587 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:29.609789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31517e-f335-4952-ae3e-b0ccd6d89568 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:29.654838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3ad13-c878-4474-a37f-4bec1f259f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:29.655507 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3ad13-c878-4474-a37f-4bec1f259f67 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:29.655848 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3ad13-c878-4474-a37f-4bec1f259f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:29.656129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3ad13-c878-4474-a37f-4bec1f259f67 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:29.657063 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef3ad13-c878-4474-a37f-4bec1f259f67 None admin] RBAC: Adding query filter params (role.id=73fb34f5dbe1404fa7ec015422951829, scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c, user.id=57c664aabb3c487e88c8884d7ede941f) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:29.657375 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef3ad13-c878-4474-a37f-4bec1f259f67 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:29.658821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef3ad13-c878-4474-a37f-4bec1f259f67 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:29.670135 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 104/208] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:29 2022] GET /identity/v3/role_assignments?user.id=57c664aabb3c487e88c8884d7ede941f&scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c&role.id=73fb34f5dbe1404fa7ec015422951829 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:30.256966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286222d8-401d-4dd3-843a-f08d92c14143 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:30.257873 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286222d8-401d-4dd3-843a-f08d92c14143 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:30.258350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286222d8-401d-4dd3-843a-f08d92c14143 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:30.259929 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 105/209] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:30.265705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fa940-ddcf-4cc5-9bac-b8a24aafab4e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:30.266561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fa940-ddcf-4cc5-9bac-b8a24aafab4e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:30.267064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31fa940-ddcf-4cc5-9bac-b8a24aafab4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:30.289664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c31fa940-ddcf-4cc5-9bac-b8a24aafab4e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:30.291171 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c31fa940-ddcf-4cc5-9bac-b8a24aafab4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:30.304235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df86f1a-d225-4dfa-9859-58e260127f57 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:30.304539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df86f1a-d225-4dfa-9859-58e260127f57 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:30.304795 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df86f1a-d225-4dfa-9859-58e260127f57 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:30.306200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 106/210] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:30.312146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654d0ed-2459-4667-87f0-6b11b32cd512 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:30.312438 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654d0ed-2459-4667-87f0-6b11b32cd512 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:30.312715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654d0ed-2459-4667-87f0-6b11b32cd512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:30.331188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5654d0ed-2459-4667-87f0-6b11b32cd512 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:30.332393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5654d0ed-2459-4667-87f0-6b11b32cd512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:30.370566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 105/211] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:30.394605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fa940-ddcf-4cc5-9bac-b8a24aafab4e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:30.413041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 107/212] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:30.428676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654d0ed-2459-4667-87f0-6b11b32cd512 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:30.437856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:30.481286 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:30.500525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31fa940-ddcf-4cc5-9bac-b8a24aafab4e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:30.536048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653dfaaf-f59a-44f9-aec7-3c59a13b6a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:30.536849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653dfaaf-f59a-44f9-aec7-3c59a13b6a7d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:30.537175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653dfaaf-f59a-44f9-aec7-3c59a13b6a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:30.537447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653dfaaf-f59a-44f9-aec7-3c59a13b6a7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:30.537862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654d0ed-2459-4667-87f0-6b11b32cd512 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:30.538670 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653dfaaf-f59a-44f9-aec7-3c59a13b6a7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:30.540113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653dfaaf-f59a-44f9-aec7-3c59a13b6a7d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:30.546445 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 106/213] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:00:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:30.547431 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653dfaaf-f59a-44f9-aec7-3c59a13b6a7d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:30.575863 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc40026-7ff0-432f-b6ac-208c2a15382c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:30.577427 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc40026-7ff0-432f-b6ac-208c2a15382c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:30.577917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc40026-7ff0-432f-b6ac-208c2a15382c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:30.578411 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc40026-7ff0-432f-b6ac-208c2a15382c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:30.589675 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc40026-7ff0-432f-b6ac-208c2a15382c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-2dc40026-7ff0-432f-b6ac-208c2a15382c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:30.598096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:30.598983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:30.599788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:30.599788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:30.599788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:30.599788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:30.599788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:30.599788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:30.599788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:30.599788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:30.599788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:30.599788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:30.599788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:30.599788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:30.607643 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 108/214] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:30.609248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc40026-7ff0-432f-b6ac-208c2a15382c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:30.651080 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653dfaaf-f59a-44f9-aec7-3c59a13b6a7d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:30.659161 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:30.694365 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6692494-0e3c-4c96-bb36-ceb51331a226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:30.694797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6692494-0e3c-4c96-bb36-ceb51331a226 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:30.695088 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6692494-0e3c-4c96-bb36-ceb51331a226 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:30.695366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6692494-0e3c-4c96-bb36-ceb51331a226 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:30.696113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6692494-0e3c-4c96-bb36-ceb51331a226 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:30.697891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6692494-0e3c-4c96-bb36-ceb51331a226 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:30.722372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc40026-7ff0-432f-b6ac-208c2a15382c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:30.722981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 107/215] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:00:30 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:00:30.732224 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:30.774726 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dede138-c2d0-41e0-bea5-4a91e1dbbe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:30.775987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dede138-c2d0-41e0-bea5-4a91e1dbbe1d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:30.776459 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dede138-c2d0-41e0-bea5-4a91e1dbbe1d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:30.776915 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dede138-c2d0-41e0-bea5-4a91e1dbbe1d None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:30.777943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dede138-c2d0-41e0-bea5-4a91e1dbbe1d None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:30.778440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dede138-c2d0-41e0-bea5-4a91e1dbbe1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:30.780162 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dede138-c2d0-41e0-bea5-4a91e1dbbe1d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:30.790697 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 109/216] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:30.801105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6692494-0e3c-4c96-bb36-ceb51331a226 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:30.871226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2c901-96c2-46ef-baaa-c11560121f1a None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:30.871513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2c901-96c2-46ef-baaa-c11560121f1a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:30.871795 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2c901-96c2-46ef-baaa-c11560121f1a None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:30.873377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 110/217] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:00:30.878243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa5436-bc6f-4ed2-a92c-d2b9e3a5444b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:30.878532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa5436-bc6f-4ed2-a92c-d2b9e3a5444b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:30.878811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa5436-bc6f-4ed2-a92c-d2b9e3a5444b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:30.897638 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-23fa5436-bc6f-4ed2-a92c-d2b9e3a5444b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:30.898879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-23fa5436-bc6f-4ed2-a92c-d2b9e3a5444b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:30.919202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6692494-0e3c-4c96-bb36-ceb51331a226 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:30.969691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36a77b-efe3-47c1-96a4-351848413d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:30.970573 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36a77b-efe3-47c1-96a4-351848413d63 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:30.970860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36a77b-efe3-47c1-96a4-351848413d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:30.971184 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36a77b-efe3-47c1-96a4-351848413d63 None admin] PATH_INFO: `/v3/users/99bf6c84740b4c6d9ab9ea27dba9081c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:30.989038 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 111/218] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:30.993547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36a77b-efe3-47c1-96a4-351848413d63 None admin] RBAC: Authorizing `identity:get_user(user_id=99bf6c84740b4c6d9ab9ea27dba9081c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:30.995065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36a77b-efe3-47c1-96a4-351848413d63 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:30.998545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 108/219] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:30 2022] GET /identity/v3/users/99bf6c84740b4c6d9ab9ea27dba9081c => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:31.003182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa5436-bc6f-4ed2-a92c-d2b9e3a5444b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:31.012292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36a77b-efe3-47c1-96a4-351848413d63 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:31.037070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:31.127358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:31.131527 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa5436-bc6f-4ed2-a92c-d2b9e3a5444b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:31.144134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36a77b-efe3-47c1-96a4-351848413d63 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:31.173784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b143ea-e9eb-40d8-8fff-0ad057c7defe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:31.175049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b143ea-e9eb-40d8-8fff-0ad057c7defe None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:31.175968 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b143ea-e9eb-40d8-8fff-0ad057c7defe None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:31.176661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b143ea-e9eb-40d8-8fff-0ad057c7defe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:31.181790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dcc83-1b61-4bb7-8032-00e201551f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:31.183199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dcc83-1b61-4bb7-8032-00e201551f71 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:31.183547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dcc83-1b61-4bb7-8032-00e201551f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:31.183886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471dcc83-1b61-4bb7-8032-00e201551f71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:31.205984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471dcc83-1b61-4bb7-8032-00e201551f71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-471dcc83-1b61-4bb7-8032-00e201551f71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:31.222499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:31.223472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:31.224267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:31.224267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:31.224267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:31.224267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:31.224267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:31.224267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:31.224267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:31.224267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:31.224267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:31.224267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:31.224267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:31.224267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:31.226550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 109/220] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:00:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:31.230441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dcc83-1b61-4bb7-8032-00e201551f71 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:31.286584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b143ea-e9eb-40d8-8fff-0ad057c7defe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:31.288117 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b143ea-e9eb-40d8-8fff-0ad057c7defe None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:31.381320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:31.421322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471dcc83-1b61-4bb7-8032-00e201551f71 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:31.448048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:31.453968 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 112/221] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:31.459607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b143ea-e9eb-40d8-8fff-0ad057c7defe None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:31.481624 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fa566-e8fb-45e1-94f1-1ef51306cc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:31.482363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fa566-e8fb-45e1-94f1-1ef51306cc20 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:31.482643 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fa566-e8fb-45e1-94f1-1ef51306cc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:31.482913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4fa566-e8fb-45e1-94f1-1ef51306cc20 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:31.483740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4fa566-e8fb-45e1-94f1-1ef51306cc20 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:31.484045 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4fa566-e8fb-45e1-94f1-1ef51306cc20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:31.485465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4fa566-e8fb-45e1-94f1-1ef51306cc20 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:31.506592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 110/222] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:31.513930 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fa566-e8fb-45e1-94f1-1ef51306cc20 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:31.577575 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b143ea-e9eb-40d8-8fff-0ad057c7defe None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:31.644749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e85cc-a019-40d8-823f-5f939bdde3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:31.645661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e85cc-a019-40d8-823f-5f939bdde3ca None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:31.645943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e85cc-a019-40d8-823f-5f939bdde3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:31.646217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028e85cc-a019-40d8-823f-5f939bdde3ca None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:31.647438 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e85cc-a019-40d8-823f-5f939bdde3ca None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:31.648879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028e85cc-a019-40d8-823f-5f939bdde3ca None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:31.655782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 113/223] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:31 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:31.656616 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e85cc-a019-40d8-823f-5f939bdde3ca None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:31.737308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:31.770754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028e85cc-a019-40d8-823f-5f939bdde3ca None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:31.775103 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4fa566-e8fb-45e1-94f1-1ef51306cc20 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:31.812041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f1065-ebc8-411f-9dc9-ecb3854a44d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:31.812969 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f1065-ebc8-411f-9dc9-ecb3854a44d5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:31.813230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f1065-ebc8-411f-9dc9-ecb3854a44d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:31.813512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f1065-ebc8-411f-9dc9-ecb3854a44d5 None admin] PATH_INFO: `/v3/users/c255f73da79249a2b651aa65b5184f59` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:31.814565 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43372f80-595a-4fb2-8df9-67a70963afaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:31.814727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f1065-ebc8-411f-9dc9-ecb3854a44d5 None admin] RBAC: Authorizing `identity:get_user(user_id=c255f73da79249a2b651aa65b5184f59)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:31.815518 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43372f80-595a-4fb2-8df9-67a70963afaa None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:31.815939 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43372f80-595a-4fb2-8df9-67a70963afaa None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:31.816187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f1065-ebc8-411f-9dc9-ecb3854a44d5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:31.816492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43372f80-595a-4fb2-8df9-67a70963afaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:31.817492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43372f80-595a-4fb2-8df9-67a70963afaa None admin] RBAC: Adding query filter params (role.id=3d9688bbbc15422cbfc59ded082fbe41, scope.project.id=00a316c6194d45b392adff3ea7182a1c, user.id=99bf6c84740b4c6d9ab9ea27dba9081c) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:31.817964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43372f80-595a-4fb2-8df9-67a70963afaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:31.818234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 111/224] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:31 2022] GET /identity/v3/users/c255f73da79249a2b651aa65b5184f59 => generated 283 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:31.823870 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b259e-6e20-4eca-8770-242b61679a33 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:31.823870 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b259e-6e20-4eca-8770-242b61679a33 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:31.823870 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809b259e-6e20-4eca-8770-242b61679a33 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:31.824128 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43372f80-595a-4fb2-8df9-67a70963afaa None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:31.824212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 112/225] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:31.825665 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b259e-6e20-4eca-8770-242b61679a33 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:31.832743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 114/226] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:31 2022] GET /identity/v3/role_assignments?user.id=99bf6c84740b4c6d9ab9ea27dba9081c&scope.project.id=00a316c6194d45b392adff3ea7182a1c&role.id=3d9688bbbc15422cbfc59ded082fbe41 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:31.833864 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e106f2b-c33a-4e7d-870d-a6b5773ae87d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:31.834154 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e106f2b-c33a-4e7d-870d-a6b5773ae87d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:31.834437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e106f2b-c33a-4e7d-870d-a6b5773ae87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:31.844183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:31.854595 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5e106f2b-c33a-4e7d-870d-a6b5773ae87d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:31.855913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5e106f2b-c33a-4e7d-870d-a6b5773ae87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:31.955529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809b259e-6e20-4eca-8770-242b61679a33 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:31.990596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 115/227] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:32.007515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80188be0-8d85-4dca-889f-38f4d623bf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:32.008658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80188be0-8d85-4dca-889f-38f4d623bf63 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:32.009169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80188be0-8d85-4dca-889f-38f4d623bf63 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:32.009689 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80188be0-8d85-4dca-889f-38f4d623bf63 None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:32.011070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80188be0-8d85-4dca-889f-38f4d623bf63 None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:32.012718 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80188be0-8d85-4dca-889f-38f4d623bf63 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:32.014924 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 113/228] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:31 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:32.020729 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e106f2b-c33a-4e7d-870d-a6b5773ae87d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:32.024624 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80188be0-8d85-4dca-889f-38f4d623bf63 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:32.141362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:32.152265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:32.172791 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80188be0-8d85-4dca-889f-38f4d623bf63 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:32.209834 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf36526-5e6f-4846-887a-c8626eee2e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:32.210980 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf36526-5e6f-4846-887a-c8626eee2e1a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:32.211459 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf36526-5e6f-4846-887a-c8626eee2e1a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:32.211951 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf36526-5e6f-4846-887a-c8626eee2e1a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:32.223832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf36526-5e6f-4846-887a-c8626eee2e1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-1bf36526-5e6f-4846-887a-c8626eee2e1a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:32.232841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:32.233778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:32.234577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:32.234577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:32.234577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:32.234577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:32.234577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:32.234577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:32.234577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:32.234577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:32.234577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:32.234577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:32.234577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:32.234577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:32.237315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 114/229] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:00:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:00:32.254315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf36526-5e6f-4846-887a-c8626eee2e1a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:32.255472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e106f2b-c33a-4e7d-870d-a6b5773ae87d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:32.296714 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b7434-3d83-4f86-8edd-1b8afb70da5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:32.298591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b7434-3d83-4f86-8edd-1b8afb70da5d None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:32.299107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b7434-3d83-4f86-8edd-1b8afb70da5d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:32.299589 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52b7434-3d83-4f86-8edd-1b8afb70da5d None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c/users/c255f73da79249a2b651aa65b5184f59/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:32.302108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b7434-3d83-4f86-8edd-1b8afb70da5d None admin] RBAC: Authorizing `identity:create_grant(project_id=00a316c6194d45b392adff3ea7182a1c, user_id=c255f73da79249a2b651aa65b5184f59, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:32.304048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52b7434-3d83-4f86-8edd-1b8afb70da5d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:32.322455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 116/230] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:00:32 2022] PUT /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c/users/c255f73da79249a2b651aa65b5184f59/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:00:32.423543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf36526-5e6f-4846-887a-c8626eee2e1a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:32.463661 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d293e-267f-4650-acb4-d245ea7e9ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:32.464713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d293e-267f-4650-acb4-d245ea7e9ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:32.465228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d293e-267f-4650-acb4-d245ea7e9ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:32.465807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7d293e-267f-4650-acb4-d245ea7e9ed0 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:32.466806 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d293e-267f-4650-acb4-d245ea7e9ed0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:32.467319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d293e-267f-4650-acb4-d245ea7e9ed0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:32.469077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7d293e-267f-4650-acb4-d245ea7e9ed0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:32.475373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:32.479439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 115/231] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:00:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:32.485995 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b7434-3d83-4f86-8edd-1b8afb70da5d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:32.590160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52b7434-3d83-4f86-8edd-1b8afb70da5d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:32.610897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:32.625651 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f6ed1-c30b-44e3-a4e0-05bba7d09b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:32.626870 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f6ed1-c30b-44e3-a4e0-05bba7d09b85 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:32.627340 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f6ed1-c30b-44e3-a4e0-05bba7d09b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:32.627617 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7f6ed1-c30b-44e3-a4e0-05bba7d09b85 None admin] PATH_INFO: `/v3/users/c255f73da79249a2b651aa65b5184f59` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:32.628864 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f6ed1-c30b-44e3-a4e0-05bba7d09b85 None admin] RBAC: Authorizing `identity:get_user(user_id=c255f73da79249a2b651aa65b5184f59)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:32.630406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7f6ed1-c30b-44e3-a4e0-05bba7d09b85 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:32.632071 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 117/232] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:32 2022] GET /identity/v3/users/c255f73da79249a2b651aa65b5184f59 => generated 283 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:32.637726 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d293e-267f-4650-acb4-d245ea7e9ed0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:32.747351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7d293e-267f-4650-acb4-d245ea7e9ed0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:32.783853 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e36140-3b76-412f-a9b2-3aaf9aa46055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:32.784725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e36140-3b76-412f-a9b2-3aaf9aa46055 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:32.785046 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e36140-3b76-412f-a9b2-3aaf9aa46055 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:32.785326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e36140-3b76-412f-a9b2-3aaf9aa46055 None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:32.786528 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e36140-3b76-412f-a9b2-3aaf9aa46055 None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:32.787976 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e36140-3b76-412f-a9b2-3aaf9aa46055 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:32.790065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 116/233] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:32 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:32.798182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f6ed1-c30b-44e3-a4e0-05bba7d09b85 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:32.935192 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7f6ed1-c30b-44e3-a4e0-05bba7d09b85 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:32.978639 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e5c02-ffa8-4669-9c06-35d036f79e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:32.979579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e5c02-ffa8-4669-9c06-35d036f79e66 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:32.979860 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e5c02-ffa8-4669-9c06-35d036f79e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:32.980133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e5c02-ffa8-4669-9c06-35d036f79e66 None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c/users/c255f73da79249a2b651aa65b5184f59/roles/f4e1d6f34f674f539ea7f2564646c0a5` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:32.983522 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e5c02-ffa8-4669-9c06-35d036f79e66 None admin] RBAC: Authorizing `identity:create_grant(project_id=00a316c6194d45b392adff3ea7182a1c, user_id=c255f73da79249a2b651aa65b5184f59, role_id=f4e1d6f34f674f539ea7f2564646c0a5)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:32.984998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e5c02-ffa8-4669-9c06-35d036f79e66 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:33.026204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 118/234] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:00:32 2022] PUT /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c/users/c255f73da79249a2b651aa65b5184f59/roles/f4e1d6f34f674f539ea7f2564646c0a5 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:00:33.324555 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12fb1d-07a3-4785-934d-a108f0f9e5d2 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:33.324756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12fb1d-07a3-4785-934d-a108f0f9e5d2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:33.325300 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12fb1d-07a3-4785-934d-a108f0f9e5d2 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:33.326904 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 117/235] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:33.333073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eca464-0986-4e0f-a449-a1f05071f2bb None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:33.333703 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eca464-0986-4e0f-a449-a1f05071f2bb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:33.333703 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eca464-0986-4e0f-a449-a1f05071f2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:33.353897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-00eca464-0986-4e0f-a449-a1f05071f2bb None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:33.355091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-00eca464-0986-4e0f-a449-a1f05071f2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:33.407744 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:33.439062 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 119/236] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:33.464289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12fb1d-07a3-4785-934d-a108f0f9e5d2 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:33.470195 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560eaebe-5ada-4095-91da-cedf5ef5559c None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:33.470491 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560eaebe-5ada-4095-91da-cedf5ef5559c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:33.470769 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560eaebe-5ada-4095-91da-cedf5ef5559c None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:33.472347 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 120/237] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:00:33.477208 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e8cde-f865-489a-b819-1e12fb0f6139 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:33.477493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e8cde-f865-489a-b819-1e12fb0f6139 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:33.477835 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5e8cde-f865-489a-b819-1e12fb0f6139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:33.505402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6e5e8cde-f865-489a-b819-1e12fb0f6139 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:33.506595 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6e5e8cde-f865-489a-b819-1e12fb0f6139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:33.587541 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:33.631468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12fb1d-07a3-4785-934d-a108f0f9e5d2 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:33.656813 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 121/238] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:33.688548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3407fe0-db3b-4dbc-82ce-b24a5a364847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:33.689400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3407fe0-db3b-4dbc-82ce-b24a5a364847 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:33.689715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3407fe0-db3b-4dbc-82ce-b24a5a364847 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:33.689991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3407fe0-db3b-4dbc-82ce-b24a5a364847 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:33.691189 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3407fe0-db3b-4dbc-82ce-b24a5a364847 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:33.692666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3407fe0-db3b-4dbc-82ce-b24a5a364847 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:33.694356 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 118/239] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:00:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:33.700645 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e8cde-f865-489a-b819-1e12fb0f6139 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:33.713105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3407fe0-db3b-4dbc-82ce-b24a5a364847 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:33.848333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3407fe0-db3b-4dbc-82ce-b24a5a364847 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:33.920213 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd59aeb-ee02-44f0-9695-550d3e22e528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:33.922356 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5e8cde-f865-489a-b819-1e12fb0f6139 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:33.923347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd59aeb-ee02-44f0-9695-550d3e22e528 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:33.925109 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd59aeb-ee02-44f0-9695-550d3e22e528 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:33.925579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd59aeb-ee02-44f0-9695-550d3e22e528 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:33.931559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:33.944524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:33.946085 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd59aeb-ee02-44f0-9695-550d3e22e528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-cbd59aeb-ee02-44f0-9695-550d3e22e528 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:33.959612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:33.960527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:33.961367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:33.961367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:33.961367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:33.961367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:33.961367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:33.961367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:33.961367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:33.961367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:33.961367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:33.961367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:33.961367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:33.961367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:33.961367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 119/240] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:33.967597 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd59aeb-ee02-44f0-9695-550d3e22e528 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:33.978226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508a903-6376-4c35-98d6-2d73d8b2dcc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:33.984508 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508a903-6376-4c35-98d6-2d73d8b2dcc3 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:33.985071 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508a903-6376-4c35-98d6-2d73d8b2dcc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:33.985511 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1508a903-6376-4c35-98d6-2d73d8b2dcc3 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:33.986415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1508a903-6376-4c35-98d6-2d73d8b2dcc3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:33.988038 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1508a903-6376-4c35-98d6-2d73d8b2dcc3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:34.031677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 122/241] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:00:33 2022] POST /identity/v3/users => generated 293 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:00:34.107110 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd59aeb-ee02-44f0-9695-550d3e22e528 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:34.145298 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc438b-98e3-4d4d-a2fa-d5bb81b16237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:34.146326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc438b-98e3-4d4d-a2fa-d5bb81b16237 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:34.146801 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc438b-98e3-4d4d-a2fa-d5bb81b16237 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:34.147264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc438b-98e3-4d4d-a2fa-d5bb81b16237 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:34.148242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc438b-98e3-4d4d-a2fa-d5bb81b16237 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:34.148740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc438b-98e3-4d4d-a2fa-d5bb81b16237 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:34.151115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc438b-98e3-4d4d-a2fa-d5bb81b16237 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:34.161269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 120/242] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:34.170149 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508a903-6376-4c35-98d6-2d73d8b2dcc3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:34.297921 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1508a903-6376-4c35-98d6-2d73d8b2dcc3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:34.335471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26650b-fa13-4afa-ac5b-64e0d3042671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:34.336364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26650b-fa13-4afa-ac5b-64e0d3042671 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:34.336793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26650b-fa13-4afa-ac5b-64e0d3042671 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:34.337110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26650b-fa13-4afa-ac5b-64e0d3042671 None admin] PATH_INFO: `/v3/users/a215c7f3d80941148fb04eb5b0fcc7c9` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:34.351510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26650b-fa13-4afa-ac5b-64e0d3042671 None admin] RBAC: Authorizing `identity:get_user(user_id=a215c7f3d80941148fb04eb5b0fcc7c9)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:34.353597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26650b-fa13-4afa-ac5b-64e0d3042671 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:34.355705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 123/243] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:34 2022] GET /identity/v3/users/a215c7f3d80941148fb04eb5b0fcc7c9 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:34.360819 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc438b-98e3-4d4d-a2fa-d5bb81b16237 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:34.396637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f49707-c315-437e-ac13-938eb003fa9e None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:34.397113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f49707-c315-437e-ac13-938eb003fa9e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:34.397526 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f49707-c315-437e-ac13-938eb003fa9e None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:34.399014 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 124/244] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:34.406321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147e4bb-3107-43ae-a83b-0ba391485985 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:34.406736 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147e4bb-3107-43ae-a83b-0ba391485985 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:34.407274 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147e4bb-3107-43ae-a83b-0ba391485985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:34.424915 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2147e4bb-3107-43ae-a83b-0ba391485985 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:34.426466 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2147e4bb-3107-43ae-a83b-0ba391485985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:34.444865 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:34.446868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:34.502441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc438b-98e3-4d4d-a2fa-d5bb81b16237 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:34.509023 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 125/245] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:34.536003 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147e4bb-3107-43ae-a83b-0ba391485985 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:34.540872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6258c-d92c-47f8-bb46-0c182e87feab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:34.541600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6258c-d92c-47f8-bb46-0c182e87feab None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:34.541877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6258c-d92c-47f8-bb46-0c182e87feab None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:34.542147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6258c-d92c-47f8-bb46-0c182e87feab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:34.633609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6258c-d92c-47f8-bb46-0c182e87feab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:34.635132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6258c-d92c-47f8-bb46-0c182e87feab None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:34.652156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147e4bb-3107-43ae-a83b-0ba391485985 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:34.687882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa4918-2157-47c0-9ec1-733c60882ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:34.688696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa4918-2157-47c0-9ec1-733c60882ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:34.689009 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa4918-2157-47c0-9ec1-733c60882ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:34.689290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aa4918-2157-47c0-9ec1-733c60882ff9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:34.701228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aa4918-2157-47c0-9ec1-733c60882ff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-b3aa4918-2157-47c0-9ec1-733c60882ff9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:34.709088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:34.709981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:34.710854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:34.710854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:34.710854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:34.710854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:34.710854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:34.710854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:34.710854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:34.710854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:34.710854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:34.710854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:34.710854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:34.710854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:34.712625 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 126/246] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:34.717179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa4918-2157-47c0-9ec1-733c60882ff9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:34.717558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:34.734868 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:34.748730 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 121/247] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:34.754736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6258c-d92c-47f8-bb46-0c182e87feab None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:34.829426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aa4918-2157-47c0-9ec1-733c60882ff9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:34.866854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307f46f-e5d8-418a-8a55-d8e54f9ba977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:34.867051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6258c-d92c-47f8-bb46-0c182e87feab None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:34.868482 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307f46f-e5d8-418a-8a55-d8e54f9ba977 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:34.868969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307f46f-e5d8-418a-8a55-d8e54f9ba977 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:34.869438 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307f46f-e5d8-418a-8a55-d8e54f9ba977 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:34.870386 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307f46f-e5d8-418a-8a55-d8e54f9ba977 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:34.870886 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307f46f-e5d8-418a-8a55-d8e54f9ba977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:34.872449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307f46f-e5d8-418a-8a55-d8e54f9ba977 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:34.881447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 127/248] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:34.886835 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46781073-735b-4033-bab2-d842f840c424 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:34.887329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46781073-735b-4033-bab2-d842f840c424 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:34.887795 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46781073-735b-4033-bab2-d842f840c424 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:34.889320 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 128/249] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:34.892516 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46781073-735b-4033-bab2-d842f840c424 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:34.904385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fa17b-d451-4fbf-be0b-edb92fab3319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:34.905375 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fa17b-d451-4fbf-be0b-edb92fab3319 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:34.905649 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fa17b-d451-4fbf-be0b-edb92fab3319 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:34.905915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fa17b-d451-4fbf-be0b-edb92fab3319 None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:34.907265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fa17b-d451-4fbf-be0b-edb92fab3319 None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:34.908748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fa17b-d451-4fbf-be0b-edb92fab3319 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:34.911812 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 122/250] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:34 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:34.917884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c26d7-5881-41ee-85d7-5ad6bd019069 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:34.917884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c26d7-5881-41ee-85d7-5ad6bd019069 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:34.917884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c26d7-5881-41ee-85d7-5ad6bd019069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:34.943901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e26c26d7-5881-41ee-85d7-5ad6bd019069 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:34.945116 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e26c26d7-5881-41ee-85d7-5ad6bd019069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:34.988322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:35.036321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:35.081961 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 123/251] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:35.083014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c26d7-5881-41ee-85d7-5ad6bd019069 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:35.087663 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46781073-735b-4033-bab2-d842f840c424 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:35.158324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf05332-98a3-470f-b0df-c87212d1b47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:35.159267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf05332-98a3-470f-b0df-c87212d1b47b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:35.159549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf05332-98a3-470f-b0df-c87212d1b47b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:35.159824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf05332-98a3-470f-b0df-c87212d1b47b None admin] PATH_INFO: `/v3/users/99bf6c84740b4c6d9ab9ea27dba9081c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:35.163001 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf05332-98a3-470f-b0df-c87212d1b47b None admin] RBAC: Authorizing `identity:get_user(user_id=99bf6c84740b4c6d9ab9ea27dba9081c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:35.164603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf05332-98a3-470f-b0df-c87212d1b47b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:35.166946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 129/252] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:34 2022] GET /identity/v3/users/99bf6c84740b4c6d9ab9ea27dba9081c => generated 297 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:35.167758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf05332-98a3-470f-b0df-c87212d1b47b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:35.241722 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c26d7-5881-41ee-85d7-5ad6bd019069 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:35.280638 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62ac8a-c89f-4dd4-80fd-3bfa91e3eeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:35.281379 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62ac8a-c89f-4dd4-80fd-3bfa91e3eeb5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:35.281663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62ac8a-c89f-4dd4-80fd-3bfa91e3eeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:35.281936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62ac8a-c89f-4dd4-80fd-3bfa91e3eeb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:35.282774 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62ac8a-c89f-4dd4-80fd-3bfa91e3eeb5 None admin] RBAC: Adding query filter params (role.id=73fb34f5dbe1404fa7ec015422951829, scope.project.id=00a316c6194d45b392adff3ea7182a1c, user.id=a215c7f3d80941148fb04eb5b0fcc7c9) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:35.283523 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62ac8a-c89f-4dd4-80fd-3bfa91e3eeb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:35.285053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62ac8a-c89f-4dd4-80fd-3bfa91e3eeb5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:35.290963 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 124/253] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:35 2022] GET /identity/v3/role_assignments?user.id=a215c7f3d80941148fb04eb5b0fcc7c9&scope.project.id=00a316c6194d45b392adff3ea7182a1c&role.id=73fb34f5dbe1404fa7ec015422951829 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:35.291763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62ac8a-c89f-4dd4-80fd-3bfa91e3eeb5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:35.318367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:35.407337 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62ac8a-c89f-4dd4-80fd-3bfa91e3eeb5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:35.442707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecec5a-8134-493d-a718-bd274999c6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:35.443490 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecec5a-8134-493d-a718-bd274999c6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:35.443764 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecec5a-8134-493d-a718-bd274999c6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:35.444031 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecec5a-8134-493d-a718-bd274999c6c0 None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:35.445264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ecec5a-8134-493d-a718-bd274999c6c0 None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:35.447262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ecec5a-8134-493d-a718-bd274999c6c0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:35.449330 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 125/254] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:35 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:35.454981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecec5a-8134-493d-a718-bd274999c6c0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:35.482987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf05332-98a3-470f-b0df-c87212d1b47b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:35.557943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:35.571050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecec5a-8134-493d-a718-bd274999c6c0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:35.609525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60658d2f-c682-4772-99b6-78c5dca821ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:35.609889 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60658d2f-c682-4772-99b6-78c5dca821ea None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:35.610181 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60658d2f-c682-4772-99b6-78c5dca821ea None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:35.610452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60658d2f-c682-4772-99b6-78c5dca821ea None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c/users/99bf6c84740b4c6d9ab9ea27dba9081c/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:35.612414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60658d2f-c682-4772-99b6-78c5dca821ea None admin] RBAC: Authorizing `identity:create_grant(project_id=00a316c6194d45b392adff3ea7182a1c, user_id=99bf6c84740b4c6d9ab9ea27dba9081c, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:35.613631 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5221739-e3d2-488d-830f-d551a80ac5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:35.613959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60658d2f-c682-4772-99b6-78c5dca821ea None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:35.614449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5221739-e3d2-488d-830f-d551a80ac5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:35.614731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5221739-e3d2-488d-830f-d551a80ac5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:35.615005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5221739-e3d2-488d-830f-d551a80ac5e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:35.633475 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5221739-e3d2-488d-830f-d551a80ac5e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:35.636016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 126/255] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:00:35 2022] PUT /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c/users/99bf6c84740b4c6d9ab9ea27dba9081c/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-f5221739-e3d2-488d-830f-d551a80ac5e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:35.646775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:35.648402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:35.650261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:35.650261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:35.650261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:35.650261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:35.650261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:35.650261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:35.650261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:35.650261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:35.650261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:35.650261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:35.650261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:35.650261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:35.653142 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 130/256] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:00:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 486 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:35.660785 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60658d2f-c682-4772-99b6-78c5dca821ea None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:35.666784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf71ab-decb-4c9a-b3c7-1fed1e4fcfde None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:35.667067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf71ab-decb-4c9a-b3c7-1fed1e4fcfde None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:35.667346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf71ab-decb-4c9a-b3c7-1fed1e4fcfde None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:35.669077 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 131/257] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:00:35.674375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05de7-70e8-46fa-8d9e-6c34bf5937b0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:35.674657 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05de7-70e8-46fa-8d9e-6c34bf5937b0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:35.674937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab05de7-70e8-46fa-8d9e-6c34bf5937b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:35.687436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:35.711780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7ab05de7-70e8-46fa-8d9e-6c34bf5937b0 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:35.713062 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7ab05de7-70e8-46fa-8d9e-6c34bf5937b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:35.783042 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60658d2f-c682-4772-99b6-78c5dca821ea None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:35.825274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f66a8-8233-4510-b663-5106cdfa5a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:35.826050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f66a8-8233-4510-b663-5106cdfa5a3c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:35.826551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f66a8-8233-4510-b663-5106cdfa5a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:35.826953 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945f66a8-8233-4510-b663-5106cdfa5a3c None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:35.827715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f66a8-8233-4510-b663-5106cdfa5a3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:35.828308 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f66a8-8233-4510-b663-5106cdfa5a3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:35.829859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945f66a8-8233-4510-b663-5106cdfa5a3c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:35.840717 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 127/258] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:35.850149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f66a8-8233-4510-b663-5106cdfa5a3c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:35.868800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 132/259] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:35.871011 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:35.912639 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab05de7-70e8-46fa-8d9e-6c34bf5937b0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:35.993915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945f66a8-8233-4510-b663-5106cdfa5a3c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:36.032265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb666119-ce40-463c-a588-099b37f66154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:36.033165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb666119-ce40-463c-a588-099b37f66154 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:36.033441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb666119-ce40-463c-a588-099b37f66154 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:36.034033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb666119-ce40-463c-a588-099b37f66154 None admin] PATH_INFO: `/v3/users/c255f73da79249a2b651aa65b5184f59` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:36.035474 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb666119-ce40-463c-a588-099b37f66154 None admin] RBAC: Authorizing `identity:get_user(user_id=c255f73da79249a2b651aa65b5184f59)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:36.036865 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb666119-ce40-463c-a588-099b37f66154 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:36.039435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 128/260] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:35 2022] GET /identity/v3/users/c255f73da79249a2b651aa65b5184f59 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:36.045392 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb666119-ce40-463c-a588-099b37f66154 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:36.136287 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab05de7-70e8-46fa-8d9e-6c34bf5937b0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:36.137405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:36.161096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb666119-ce40-463c-a588-099b37f66154 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:36.201315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12cf05-2860-4341-9b49-2e79a3704484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:36.202338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12cf05-2860-4341-9b49-2e79a3704484 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:36.202628 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12cf05-2860-4341-9b49-2e79a3704484 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:36.202905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12cf05-2860-4341-9b49-2e79a3704484 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:36.213346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:36.224446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29264bfa-814a-4e1d-9790-4eea656328ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:36.225696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29264bfa-814a-4e1d-9790-4eea656328ce None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:36.225988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29264bfa-814a-4e1d-9790-4eea656328ce None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:36.227018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29264bfa-814a-4e1d-9790-4eea656328ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:36.255660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29264bfa-814a-4e1d-9790-4eea656328ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-29264bfa-814a-4e1d-9790-4eea656328ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:36.270054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:36.270937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:36.271751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:36.271751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:36.271751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:36.271751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:36.271751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:36.271751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:36.271751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:36.271751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:36.271751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:36.271751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:36.271751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:36.271751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:36.271751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 133/261] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:00:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:00:36.277740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29264bfa-814a-4e1d-9790-4eea656328ce None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:36.277868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12cf05-2860-4341-9b49-2e79a3704484 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:36.279321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c12cf05-2860-4341-9b49-2e79a3704484 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:36.366973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:36.391609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 129/262] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:36.397032 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12cf05-2860-4341-9b49-2e79a3704484 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:36.499123 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29264bfa-814a-4e1d-9790-4eea656328ce None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:36.573364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad07277-2758-46b9-816c-f557267d9197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:36.574293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad07277-2758-46b9-816c-f557267d9197 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:36.574577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad07277-2758-46b9-816c-f557267d9197 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:36.574854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad07277-2758-46b9-816c-f557267d9197 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:36.575687 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad07277-2758-46b9-816c-f557267d9197 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:36.575999 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad07277-2758-46b9-816c-f557267d9197 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:36.577442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad07277-2758-46b9-816c-f557267d9197 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:36.590390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12cf05-2860-4341-9b49-2e79a3704484 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:36.599444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 134/263] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:00:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:36.609943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad07277-2758-46b9-816c-f557267d9197 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:36.678347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae525281-68e9-4853-9ee4-d96eed03fa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:36.679645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae525281-68e9-4853-9ee4-d96eed03fa1c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:36.680127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae525281-68e9-4853-9ee4-d96eed03fa1c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:36.680613 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae525281-68e9-4853-9ee4-d96eed03fa1c None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:36.682200 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae525281-68e9-4853-9ee4-d96eed03fa1c None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:36.683856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae525281-68e9-4853-9ee4-d96eed03fa1c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:36.685823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 130/264] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:36 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:36.691746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae525281-68e9-4853-9ee4-d96eed03fa1c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:36.749292 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:36.804744 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:36.822420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad07277-2758-46b9-816c-f557267d9197 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:36.893695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae525281-68e9-4853-9ee4-d96eed03fa1c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:36.910026 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a91a5-58b0-415a-b72a-8049a779627e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:36.910962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a91a5-58b0-415a-b72a-8049a779627e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:36.911247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a91a5-58b0-415a-b72a-8049a779627e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:36.911523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a91a5-58b0-415a-b72a-8049a779627e None admin] PATH_INFO: `/v3/users/c255f73da79249a2b651aa65b5184f59` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:36.915030 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a91a5-58b0-415a-b72a-8049a779627e None admin] RBAC: Authorizing `identity:get_user(user_id=c255f73da79249a2b651aa65b5184f59)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:36.916500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a91a5-58b0-415a-b72a-8049a779627e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:36.918233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 135/265] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:36 2022] GET /identity/v3/users/c255f73da79249a2b651aa65b5184f59 => generated 283 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:36.925434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a91a5-58b0-415a-b72a-8049a779627e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:36.969067 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7e149-fb1d-4b4c-a786-20efbdb3c9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:36.969791 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7e149-fb1d-4b4c-a786-20efbdb3c9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:36.970178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7e149-fb1d-4b4c-a786-20efbdb3c9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:36.970459 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7e149-fb1d-4b4c-a786-20efbdb3c9b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:36.971477 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7e149-fb1d-4b4c-a786-20efbdb3c9b1 None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=00a316c6194d45b392adff3ea7182a1c, user.id=c255f73da79249a2b651aa65b5184f59) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:36.971792 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7e149-fb1d-4b4c-a786-20efbdb3c9b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:36.973626 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7e149-fb1d-4b4c-a786-20efbdb3c9b1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:36.982889 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 131/266] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:36 2022] GET /identity/v3/role_assignments?user.id=c255f73da79249a2b651aa65b5184f59&scope.project.id=00a316c6194d45b392adff3ea7182a1c&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:37.075478 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e8a5d-d095-4c42-9e96-77d4138f0a88 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:37.075478 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e8a5d-d095-4c42-9e96-77d4138f0a88 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:37.075478 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114e8a5d-d095-4c42-9e96-77d4138f0a88 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:37.077201 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 132/267] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:37.079723 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a91a5-58b0-415a-b72a-8049a779627e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:37.098486 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627d26d-5220-4b41-826b-dc62444021b8 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:37.098787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627d26d-5220-4b41-826b-dc62444021b8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:37.099049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627d26d-5220-4b41-826b-dc62444021b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:37.111103 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:37.124955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9627d26d-5220-4b41-826b-dc62444021b8 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:37.126496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9627d26d-5220-4b41-826b-dc62444021b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:37.135982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bd4a7-eaa3-48ab-a2f7-33c4c2234a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:37.136713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bd4a7-eaa3-48ab-a2f7-33c4c2234a7a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:37.138261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bd4a7-eaa3-48ab-a2f7-33c4c2234a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:37.138261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bd4a7-eaa3-48ab-a2f7-33c4c2234a7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:37.298722 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 133/268] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:37.302986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bd4a7-eaa3-48ab-a2f7-33c4c2234a7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:37.304815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bd4a7-eaa3-48ab-a2f7-33c4c2234a7a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:37.350020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627d26d-5220-4b41-826b-dc62444021b8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:37.423355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:37.455784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 136/269] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 531 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:37.462993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bd4a7-eaa3-48ab-a2f7-33c4c2234a7a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:37.486095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627d26d-5220-4b41-826b-dc62444021b8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:37.501304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:37.527003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca45d8-1937-46ea-aa2e-1763a62ea6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:37.528131 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca45d8-1937-46ea-aa2e-1763a62ea6d0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:37.528604 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca45d8-1937-46ea-aa2e-1763a62ea6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:37.529096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abca45d8-1937-46ea-aa2e-1763a62ea6d0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:37.540165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abca45d8-1937-46ea-aa2e-1763a62ea6d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-abca45d8-1937-46ea-aa2e-1763a62ea6d0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:37.562242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:37.563169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:37.563959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:37.563959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:37.563959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:37.563959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:37.563959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:37.563959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:37.563959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:37.563959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:37.563959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:37.563959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:37.563959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:37.563959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:37.566402 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 134/270] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:37.572777 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca45d8-1937-46ea-aa2e-1763a62ea6d0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:37.593919 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bd4a7-eaa3-48ab-a2f7-33c4c2234a7a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:37.658232 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85b04e-6b60-402e-9814-cc377a8ff844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:37.659380 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85b04e-6b60-402e-9814-cc377a8ff844 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:37.659839 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85b04e-6b60-402e-9814-cc377a8ff844 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:37.660250 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85b04e-6b60-402e-9814-cc377a8ff844 None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:37.664043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85b04e-6b60-402e-9814-cc377a8ff844 None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:37.665761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85b04e-6b60-402e-9814-cc377a8ff844 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:37.667966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 137/271] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:37 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:37.673437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85b04e-6b60-402e-9814-cc377a8ff844 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:37.689610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abca45d8-1937-46ea-aa2e-1763a62ea6d0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:37.697383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:37.724550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1d00c-d339-4da1-bd9c-75a12ff788fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:37.725419 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1d00c-d339-4da1-bd9c-75a12ff788fd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:37.725826 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1d00c-d339-4da1-bd9c-75a12ff788fd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:37.726201 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1d00c-d339-4da1-bd9c-75a12ff788fd None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:37.727054 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1d00c-d339-4da1-bd9c-75a12ff788fd None admin] RBAC: Adding query filter params (name=member) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:37.727480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1d00c-d339-4da1-bd9c-75a12ff788fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:37.728975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1d00c-d339-4da1-bd9c-75a12ff788fd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:37.737742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 135/272] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:37.743740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1d00c-d339-4da1-bd9c-75a12ff788fd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:37.828732 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85b04e-6b60-402e-9814-cc377a8ff844 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:37.849302 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:37.873895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a875bf-67b6-428d-84b3-214e428eac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:37.874561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a875bf-67b6-428d-84b3-214e428eac08 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:37.874845 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a875bf-67b6-428d-84b3-214e428eac08 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:37.875123 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a875bf-67b6-428d-84b3-214e428eac08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:37.875964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a875bf-67b6-428d-84b3-214e428eac08 None admin] RBAC: Adding query filter params (role.id=f4e1d6f34f674f539ea7f2564646c0a5, scope.project.id=00a316c6194d45b392adff3ea7182a1c, user.id=c255f73da79249a2b651aa65b5184f59) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:37.876275 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a875bf-67b6-428d-84b3-214e428eac08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:37.877940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a875bf-67b6-428d-84b3-214e428eac08 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:37.883391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1d00c-d339-4da1-bd9c-75a12ff788fd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:37.888080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 138/273] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:37 2022] GET /identity/v3/role_assignments?user.id=c255f73da79249a2b651aa65b5184f59&scope.project.id=00a316c6194d45b392adff3ea7182a1c&role.id=f4e1d6f34f674f539ea7f2564646c0a5 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:37.889410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acc298-3cdb-487d-808c-c7ea7f6be1cb None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:37.889792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acc298-3cdb-487d-808c-c7ea7f6be1cb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:37.890164 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34acc298-3cdb-487d-808c-c7ea7f6be1cb None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:37.892101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 139/274] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:37.897977 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648c4c0-da7f-4fca-894b-c51df538558f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:37.898260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648c4c0-da7f-4fca-894b-c51df538558f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:37.898634 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6648c4c0-da7f-4fca-894b-c51df538558f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:37.917109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6648c4c0-da7f-4fca-894b-c51df538558f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:37.918409 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6648c4c0-da7f-4fca-894b-c51df538558f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:37.927834 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557af85-203b-4b18-aa3e-b911e8e5fb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:37.929755 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557af85-203b-4b18-aa3e-b911e8e5fb0e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:37.930212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557af85-203b-4b18-aa3e-b911e8e5fb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:37.930494 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557af85-203b-4b18-aa3e-b911e8e5fb0e None admin] PATH_INFO: `/v3/users/1320dcaabb8a4ca29940cea8eb9dcc77` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:37.951909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557af85-203b-4b18-aa3e-b911e8e5fb0e None admin] RBAC: Authorizing `identity:get_user(user_id=1320dcaabb8a4ca29940cea8eb9dcc77)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:37.954304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557af85-203b-4b18-aa3e-b911e8e5fb0e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:37.956611 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 136/275] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:37 2022] GET /identity/v3/users/1320dcaabb8a4ca29940cea8eb9dcc77 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:37.962703 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557af85-203b-4b18-aa3e-b911e8e5fb0e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:37.996062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:38.047423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 140/276] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:38.090055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648c4c0-da7f-4fca-894b-c51df538558f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:38.099943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557af85-203b-4b18-aa3e-b911e8e5fb0e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:38.148175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764d70f-fe5d-437d-a3a9-8e673e424bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:38.148847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764d70f-fe5d-437d-a3a9-8e673e424bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:38.149165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764d70f-fe5d-437d-a3a9-8e673e424bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:38.149448 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2764d70f-fe5d-437d-a3a9-8e673e424bc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:38.150300 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764d70f-fe5d-437d-a3a9-8e673e424bc5 None admin] RBAC: Adding query filter params (role.id=3d9688bbbc15422cbfc59ded082fbe41, user.id=1320dcaabb8a4ca29940cea8eb9dcc77, scope.system=all) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:38.150615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764d70f-fe5d-437d-a3a9-8e673e424bc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:38.153258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2764d70f-fe5d-437d-a3a9-8e673e424bc5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:38.160601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 137/277] 10.222.0.61 () {56 vars in 1308 bytes} [Thu Jul 14 10:00:37 2022] GET /identity/v3/role_assignments?user.id=1320dcaabb8a4ca29940cea8eb9dcc77&scope.system=all&role.id=3d9688bbbc15422cbfc59ded082fbe41 => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:38.218904 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:38.243363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6648c4c0-da7f-4fca-894b-c51df538558f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:38.292133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26514a90-1504-4331-baf6-723f55371a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:38.293010 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26514a90-1504-4331-baf6-723f55371a3b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:38.293301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26514a90-1504-4331-baf6-723f55371a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:38.293577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26514a90-1504-4331-baf6-723f55371a3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:38.294803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26514a90-1504-4331-baf6-723f55371a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:38.296282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26514a90-1504-4331-baf6-723f55371a3b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:38.300831 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 141/278] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:00:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:38.304259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764d70f-fe5d-437d-a3a9-8e673e424bc5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:38.405956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:38.423314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2764d70f-fe5d-437d-a3a9-8e673e424bc5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:38.459349 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f329de5-e75b-4c51-af53-302c650531a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:38.460939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f329de5-e75b-4c51-af53-302c650531a4 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:38.461401 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f329de5-e75b-4c51-af53-302c650531a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:38.461905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f329de5-e75b-4c51-af53-302c650531a4 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:38.462792 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f329de5-e75b-4c51-af53-302c650531a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:38.464523 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f329de5-e75b-4c51-af53-302c650531a4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:38.489921 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 138/279] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:00:38 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:00:38.509584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cf04b-a9fc-45d6-b490-4dfdb2dc8007 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:38.510095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cf04b-a9fc-45d6-b490-4dfdb2dc8007 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:38.510168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336cf04b-a9fc-45d6-b490-4dfdb2dc8007 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:38.511550 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 142/280] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:38.517426 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30afa1-1c0b-488d-8e4f-6446b9718250 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:38.517714 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30afa1-1c0b-488d-8e4f-6446b9718250 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:38.517995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30afa1-1c0b-488d-8e4f-6446b9718250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:38.536765 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-1c30afa1-1c0b-488d-8e4f-6446b9718250 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:38.538621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1c30afa1-1c0b-488d-8e4f-6446b9718250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:38.547898 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7dacc-7ea2-4c5b-9f60-eb31c5b32d97 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:38.548269 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7dacc-7ea2-4c5b-9f60-eb31c5b32d97 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:38.548463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7dacc-7ea2-4c5b-9f60-eb31c5b32d97 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:38.550251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 143/281] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:38.556494 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82407c-f1cd-4dab-8a20-f23c48f0f737 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:38.556814 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82407c-f1cd-4dab-8a20-f23c48f0f737 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:38.557135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b82407c-f1cd-4dab-8a20-f23c48f0f737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:38.575580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6b82407c-f1cd-4dab-8a20-f23c48f0f737 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:38.576825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6b82407c-f1cd-4dab-8a20-f23c48f0f737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:38.633855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 139/282] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:38.661788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30afa1-1c0b-488d-8e4f-6446b9718250 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:38.669210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 144/283] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:38.693037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82407c-f1cd-4dab-8a20-f23c48f0f737 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:38.747708 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:38.781940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30afa1-1c0b-488d-8e4f-6446b9718250 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:38.802386 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:38.819921 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada7365-1ccc-4ef8-84fb-b59a789b649f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:38.820722 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada7365-1ccc-4ef8-84fb-b59a789b649f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:38.821033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada7365-1ccc-4ef8-84fb-b59a789b649f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:38.821135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b82407c-f1cd-4dab-8a20-f23c48f0f737 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:38.821330 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada7365-1ccc-4ef8-84fb-b59a789b649f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:38.837570 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada7365-1ccc-4ef8-84fb-b59a789b649f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-eada7365-1ccc-4ef8-84fb-b59a789b649f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:38.845378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:38.846281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:38.847074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:38.847074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:38.847074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:38.847074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:38.847074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:38.847074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:38.847074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:38.847074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:38.847074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:38.847074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:38.847074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:38.847074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:38.847074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 140/284] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:00:38.854868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada7365-1ccc-4ef8-84fb-b59a789b649f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:38.863272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53d0e7-be0e-4372-8e54-9bdd0d9666a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:38.864119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53d0e7-be0e-4372-8e54-9bdd0d9666a5 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:38.864401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53d0e7-be0e-4372-8e54-9bdd0d9666a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:38.864671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53d0e7-be0e-4372-8e54-9bdd0d9666a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:38.879113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53d0e7-be0e-4372-8e54-9bdd0d9666a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-cf53d0e7-be0e-4372-8e54-9bdd0d9666a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:38.889406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:38.890531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:38.892099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:38.892099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:38.892099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:38.892099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:38.892099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:38.892099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:38.892099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:38.892099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:38.892099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:38.892099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:38.892099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:38.892099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:38.892099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 145/285] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:00:38.897796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53d0e7-be0e-4372-8e54-9bdd0d9666a5 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:38.987409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada7365-1ccc-4ef8-84fb-b59a789b649f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:39.018692 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:39.025991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d80a6c-27cc-491e-853f-af01e2748dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:39.026670 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d80a6c-27cc-491e-853f-af01e2748dbe None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:39.026947 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d80a6c-27cc-491e-853f-af01e2748dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:39.027218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d80a6c-27cc-491e-853f-af01e2748dbe None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:39.027977 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d80a6c-27cc-491e-853f-af01e2748dbe None admin] RBAC: Adding query filter params (name=member) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:39.028152 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53d0e7-be0e-4372-8e54-9bdd0d9666a5 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:39.028305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d80a6c-27cc-491e-853f-af01e2748dbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:39.029718 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d80a6c-27cc-491e-853f-af01e2748dbe None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:39.039608 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 141/286] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:39.046050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d80a6c-27cc-491e-853f-af01e2748dbe None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:39.067620 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd2ac0-2f84-45b2-b785-7adc02cd6ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:39.068676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd2ac0-2f84-45b2-b785-7adc02cd6ec8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:39.069525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd2ac0-2f84-45b2-b785-7adc02cd6ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:39.069803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd2ac0-2f84-45b2-b785-7adc02cd6ec8 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:39.070860 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd2ac0-2f84-45b2-b785-7adc02cd6ec8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:39.071166 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd2ac0-2f84-45b2-b785-7adc02cd6ec8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:39.076954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd2ac0-2f84-45b2-b785-7adc02cd6ec8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:39.082556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 146/287] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:39.087820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd2ac0-2f84-45b2-b785-7adc02cd6ec8 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:39.101281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:39.200227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd2ac0-2f84-45b2-b785-7adc02cd6ec8 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:39.264403 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152cb1d-c743-4457-a256-a83e718a9ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:39.265316 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152cb1d-c743-4457-a256-a83e718a9ac5 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:39.265597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152cb1d-c743-4457-a256-a83e718a9ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:39.265871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d152cb1d-c743-4457-a256-a83e718a9ac5 None admin] PATH_INFO: `/v3/users/a215c7f3d80941148fb04eb5b0fcc7c9` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:39.268317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152cb1d-c743-4457-a256-a83e718a9ac5 None admin] RBAC: Authorizing `identity:get_user(user_id=a215c7f3d80941148fb04eb5b0fcc7c9)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:39.269782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d152cb1d-c743-4457-a256-a83e718a9ac5 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:39.271430 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 147/288] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:39 2022] GET /identity/v3/users/a215c7f3d80941148fb04eb5b0fcc7c9 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:39.277029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152cb1d-c743-4457-a256-a83e718a9ac5 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:39.285565 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d80a6c-27cc-491e-853f-af01e2748dbe None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:39.330903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70150b24-4051-4feb-8e34-ae731b602608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:39.331785 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70150b24-4051-4feb-8e34-ae731b602608 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:39.332203 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70150b24-4051-4feb-8e34-ae731b602608 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:39.332487 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70150b24-4051-4feb-8e34-ae731b602608 None admin] PATH_INFO: `/v3/users/99bf6c84740b4c6d9ab9ea27dba9081c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:39.333865 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70150b24-4051-4feb-8e34-ae731b602608 None admin] RBAC: Authorizing `identity:get_user(user_id=99bf6c84740b4c6d9ab9ea27dba9081c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:39.335538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70150b24-4051-4feb-8e34-ae731b602608 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:39.337941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 142/289] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:39 2022] GET /identity/v3/users/99bf6c84740b4c6d9ab9ea27dba9081c => generated 297 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:39.347280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70150b24-4051-4feb-8e34-ae731b602608 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:39.443525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:39.474228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:39.480544 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d152cb1d-c743-4457-a256-a83e718a9ac5 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:39.545357 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2eab90-e56e-464e-b82a-5098e82ede4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:39.546140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2eab90-e56e-464e-b82a-5098e82ede4c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:39.546422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2eab90-e56e-464e-b82a-5098e82ede4c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:39.546698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2eab90-e56e-464e-b82a-5098e82ede4c None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:39.548040 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2eab90-e56e-464e-b82a-5098e82ede4c None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:39.551055 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70150b24-4051-4feb-8e34-ae731b602608 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:39.552331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2eab90-e56e-464e-b82a-5098e82ede4c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:39.556291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 148/290] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:39 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:39.565825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2eab90-e56e-464e-b82a-5098e82ede4c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:39.614935 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f57e1-a41f-444a-ae41-c9adabf9e129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:39.616130 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f57e1-a41f-444a-ae41-c9adabf9e129 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:39.616409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f57e1-a41f-444a-ae41-c9adabf9e129 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:39.616682 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8f57e1-a41f-444a-ae41-c9adabf9e129 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:39.730830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2eab90-e56e-464e-b82a-5098e82ede4c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:39.735721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f57e1-a41f-444a-ae41-c9adabf9e129 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:39.737448 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8f57e1-a41f-444a-ae41-c9adabf9e129 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:39.794742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39d288-2b30-4444-b108-c94f63c73dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:39.796051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39d288-2b30-4444-b108-c94f63c73dc2 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:39.796552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39d288-2b30-4444-b108-c94f63c73dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:39.797125 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b39d288-2b30-4444-b108-c94f63c73dc2 None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c/users/a215c7f3d80941148fb04eb5b0fcc7c9/roles/73fb34f5dbe1404fa7ec015422951829` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:39.800044 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39d288-2b30-4444-b108-c94f63c73dc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=00a316c6194d45b392adff3ea7182a1c, user_id=a215c7f3d80941148fb04eb5b0fcc7c9, role_id=73fb34f5dbe1404fa7ec015422951829)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:39.802691 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b39d288-2b30-4444-b108-c94f63c73dc2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:39.819988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:39.827080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 149/291] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:00:39 2022] PUT /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c/users/a215c7f3d80941148fb04eb5b0fcc7c9/roles/73fb34f5dbe1404fa7ec015422951829 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:00:39.848671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:39.913088 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 143/292] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:39.917192 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39d288-2b30-4444-b108-c94f63c73dc2 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:40.075583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b39d288-2b30-4444-b108-c94f63c73dc2 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:40.113879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e465a36-964b-4805-a26a-b4bbf2ccc275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:40.115100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e465a36-964b-4805-a26a-b4bbf2ccc275 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:40.115571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e465a36-964b-4805-a26a-b4bbf2ccc275 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:40.116015 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e465a36-964b-4805-a26a-b4bbf2ccc275 None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:40.117847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e465a36-964b-4805-a26a-b4bbf2ccc275 None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:40.119887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e465a36-964b-4805-a26a-b4bbf2ccc275 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:40.121949 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 150/293] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:39 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:40.127590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f57e1-a41f-444a-ae41-c9adabf9e129 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:40.282623 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8f57e1-a41f-444a-ae41-c9adabf9e129 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:40.334458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7386e84a-46f5-4d77-a42c-bf9a7c5c5aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:40.335041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386e84a-46f5-4d77-a42c-bf9a7c5c5aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:40.335325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386e84a-46f5-4d77-a42c-bf9a7c5c5aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:40.335603 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7386e84a-46f5-4d77-a42c-bf9a7c5c5aa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:40.336448 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386e84a-46f5-4d77-a42c-bf9a7c5c5aa4 None admin] RBAC: Adding query filter params (role.id=3d9688bbbc15422cbfc59ded082fbe41, scope.project.id=00a316c6194d45b392adff3ea7182a1c, user.id=99bf6c84740b4c6d9ab9ea27dba9081c) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:40.336762 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386e84a-46f5-4d77-a42c-bf9a7c5c5aa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:40.338226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7386e84a-46f5-4d77-a42c-bf9a7c5c5aa4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:40.344478 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 144/294] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:40 2022] GET /identity/v3/role_assignments?user.id=99bf6c84740b4c6d9ab9ea27dba9081c&scope.project.id=00a316c6194d45b392adff3ea7182a1c&role.id=3d9688bbbc15422cbfc59ded082fbe41 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:40.564274 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23228af-c0d0-428f-a2ab-7087be0db7df None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:40.564768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23228af-c0d0-428f-a2ab-7087be0db7df None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:40.564871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23228af-c0d0-428f-a2ab-7087be0db7df None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:40.566617 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 151/295] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:40.573479 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbc6d4-4787-4d68-8491-6d5947352c84 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:40.573763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbc6d4-4787-4d68-8491-6d5947352c84 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:40.574045 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dbc6d4-4787-4d68-8491-6d5947352c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:40.592412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a9dbc6d4-4787-4d68-8491-6d5947352c84 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:40.593574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a9dbc6d4-4787-4d68-8491-6d5947352c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:40.613464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:40.668312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 145/296] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:40.694068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23228af-c0d0-428f-a2ab-7087be0db7df None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:40.775839 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:40.815766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23228af-c0d0-428f-a2ab-7087be0db7df None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:40.859291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e14ec-5877-41e1-a056-688630906810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:40.860024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e14ec-5877-41e1-a056-688630906810 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:40.860305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e14ec-5877-41e1-a056-688630906810 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:40.860579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e14ec-5877-41e1-a056-688630906810 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:40.872375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e14ec-5877-41e1-a056-688630906810 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-f31e14ec-5877-41e1-a056-688630906810 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:40.880463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:40.881862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:40.882717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:40.882717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:40.882717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:40.882717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:40.882717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:40.882717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:40.882717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:40.882717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:40.882717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:40.882717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:40.882717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:40.882717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:40.887105 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 152/297] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:40.890067 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbc6d4-4787-4d68-8491-6d5947352c84 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:40.937084 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78a388-8835-4c70-a859-4c2e6c166c22 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:40.937747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78a388-8835-4c70-a859-4c2e6c166c22 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:40.938204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78a388-8835-4c70-a859-4c2e6c166c22 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:40.939752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 153/298] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:00:40.945560 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ae5b0-ac1a-4007-8828-6495ac2c51c5 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:40.946075 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ae5b0-ac1a-4007-8828-6495ac2c51c5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:40.946540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ae5b0-ac1a-4007-8828-6495ac2c51c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:40.965105 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d89ae5b0-ac1a-4007-8828-6495ac2c51c5 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:40.966460 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d89ae5b0-ac1a-4007-8828-6495ac2c51c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:41.006093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dbc6d4-4787-4d68-8491-6d5947352c84 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:41.042524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 154/299] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:41.047406 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0bf64a-e266-4e6c-8de1-b22220a5e42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:41.048088 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bf64a-e266-4e6c-8de1-b22220a5e42f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:41.048372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bf64a-e266-4e6c-8de1-b22220a5e42f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:41.048648 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0bf64a-e266-4e6c-8de1-b22220a5e42f None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:41.049493 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0bf64a-e266-4e6c-8de1-b22220a5e42f None admin] RBAC: Adding query filter params (name=member) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:41.049801 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0bf64a-e266-4e6c-8de1-b22220a5e42f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:41.051291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0bf64a-e266-4e6c-8de1-b22220a5e42f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:41.064829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 146/300] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:41.066326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ae5b0-ac1a-4007-8828-6495ac2c51c5 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:41.066879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0bf64a-e266-4e6c-8de1-b22220a5e42f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:41.087563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:41.132092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:41.176358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0bf64a-e266-4e6c-8de1-b22220a5e42f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:41.179655 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ae5b0-ac1a-4007-8828-6495ac2c51c5 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:41.212972 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae88809-ff04-4b4b-9c04-0b313059b749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:41.213777 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae88809-ff04-4b4b-9c04-0b313059b749 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:41.214056 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae88809-ff04-4b4b-9c04-0b313059b749 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:41.214337 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae88809-ff04-4b4b-9c04-0b313059b749 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:41.216699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9c859-82c9-4ead-aec8-c6cd896f492a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:41.217693 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9c859-82c9-4ead-aec8-c6cd896f492a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:41.217975 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9c859-82c9-4ead-aec8-c6cd896f492a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:41.218251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a9c859-82c9-4ead-aec8-c6cd896f492a None admin] PATH_INFO: `/v3/users/1320dcaabb8a4ca29940cea8eb9dcc77` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:41.219454 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9c859-82c9-4ead-aec8-c6cd896f492a None admin] RBAC: Authorizing `identity:get_user(user_id=1320dcaabb8a4ca29940cea8eb9dcc77)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:41.221079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a9c859-82c9-4ead-aec8-c6cd896f492a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:41.225375 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae88809-ff04-4b4b-9c04-0b313059b749 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:41.228261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 155/301] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:41 2022] GET /identity/v3/users/1320dcaabb8a4ca29940cea8eb9dcc77 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:41.229452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9c859-82c9-4ead-aec8-c6cd896f492a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-4ae88809-ff04-4b4b-9c04-0b313059b749 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:41.233338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:41.234223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:41.235004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:41.235004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:41.235004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:41.235004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:41.235004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:41.235004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:41.235004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:41.235004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:41.235004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:41.235004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:41.235004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:41.235004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:41.250315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 147/302] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:00:41.251433 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae88809-ff04-4b4b-9c04-0b313059b749 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:41.336294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:41.337117 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a9c859-82c9-4ead-aec8-c6cd896f492a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:41.361239 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae88809-ff04-4b4b-9c04-0b313059b749 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:41.367219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:41.376309 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58544d4-4bf4-4dff-bdf7-86455e261bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:41.377222 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58544d4-4bf4-4dff-bdf7-86455e261bdd None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:41.377502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58544d4-4bf4-4dff-bdf7-86455e261bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:41.377807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58544d4-4bf4-4dff-bdf7-86455e261bdd None admin] PATH_INFO: `/v3/system/users/1320dcaabb8a4ca29940cea8eb9dcc77/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:41.379241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58544d4-4bf4-4dff-bdf7-86455e261bdd None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1320dcaabb8a4ca29940cea8eb9dcc77, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:41.381184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58544d4-4bf4-4dff-bdf7-86455e261bdd None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:41.398663 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 156/303] 10.222.0.61 () {58 vars in 1334 bytes} [Thu Jul 14 10:00:41 2022] PUT /identity/v3/system/users/1320dcaabb8a4ca29940cea8eb9dcc77/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:00:41.400091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234dd5a-9816-455e-b3d7-11659870cb03 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:41.400374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234dd5a-9816-455e-b3d7-11659870cb03 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:41.400648 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8234dd5a-9816-455e-b3d7-11659870cb03 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:41.402160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 157/304] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:41.406959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcebdd3-befc-4ef2-ba9d-78b86ce825aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:41.407577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e422131-0563-4fec-8097-c1f824004837 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:41.407679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcebdd3-befc-4ef2-ba9d-78b86ce825aa None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:41.407854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e422131-0563-4fec-8097-c1f824004837 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:41.407965 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcebdd3-befc-4ef2-ba9d-78b86ce825aa None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:41.408122 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e422131-0563-4fec-8097-c1f824004837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:41.408212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcebdd3-befc-4ef2-ba9d-78b86ce825aa None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:41.408964 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcebdd3-befc-4ef2-ba9d-78b86ce825aa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:41.409266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcebdd3-befc-4ef2-ba9d-78b86ce825aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:41.410686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcebdd3-befc-4ef2-ba9d-78b86ce825aa None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:41.425777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5e422131-0563-4fec-8097-c1f824004837 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:41.426993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5e422131-0563-4fec-8097-c1f824004837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:41.427807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 148/305] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:41.434759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcebdd3-befc-4ef2-ba9d-78b86ce825aa None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:41.513323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 158/306] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:41.531837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e422131-0563-4fec-8097-c1f824004837 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:41.554159 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcebdd3-befc-4ef2-ba9d-78b86ce825aa None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:41.591223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f8aea-cd1c-4b11-a3bb-e3233a44fa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:41.592309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f8aea-cd1c-4b11-a3bb-e3233a44fa96 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:41.592689 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f8aea-cd1c-4b11-a3bb-e3233a44fa96 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:41.593115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9f8aea-cd1c-4b11-a3bb-e3233a44fa96 None admin] PATH_INFO: `/v3/users/414b398a5bb04338a6ccd9b162381d4c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:41.604514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:41.610674 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f8aea-cd1c-4b11-a3bb-e3233a44fa96 None admin] RBAC: Authorizing `identity:get_user(user_id=414b398a5bb04338a6ccd9b162381d4c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:41.612251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9f8aea-cd1c-4b11-a3bb-e3233a44fa96 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:41.615940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 149/307] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:41 2022] GET /identity/v3/users/414b398a5bb04338a6ccd9b162381d4c => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:41.621518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f8aea-cd1c-4b11-a3bb-e3233a44fa96 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:41.642436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e422131-0563-4fec-8097-c1f824004837 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:41.665576 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:41.682484 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0f956-075a-4fe7-a7c6-5dd3911be880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:41.683399 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0f956-075a-4fe7-a7c6-5dd3911be880 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:41.683741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0f956-075a-4fe7-a7c6-5dd3911be880 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:41.684073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0f956-075a-4fe7-a7c6-5dd3911be880 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:41.685405 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0f956-075a-4fe7-a7c6-5dd3911be880 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:41.686907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0f956-075a-4fe7-a7c6-5dd3911be880 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:41.688889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 159/308] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:00:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:41.695040 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0f956-075a-4fe7-a7c6-5dd3911be880 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:41.749823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9f8aea-cd1c-4b11-a3bb-e3233a44fa96 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:41.789549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4dc271-90a3-4f3b-bfe5-c5743011dbe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:41.790211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dc271-90a3-4f3b-bfe5-c5743011dbe7 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:41.790492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dc271-90a3-4f3b-bfe5-c5743011dbe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:41.790767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dc271-90a3-4f3b-bfe5-c5743011dbe7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:41.791605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4dc271-90a3-4f3b-bfe5-c5743011dbe7 None admin] RBAC: Adding query filter params (role.id=73fb34f5dbe1404fa7ec015422951829, user.id=414b398a5bb04338a6ccd9b162381d4c, scope.system=all) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:41.791915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4dc271-90a3-4f3b-bfe5-c5743011dbe7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:41.793383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4dc271-90a3-4f3b-bfe5-c5743011dbe7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:41.800337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0f956-075a-4fe7-a7c6-5dd3911be880 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:41.805065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 150/309] 10.222.0.61 () {56 vars in 1308 bytes} [Thu Jul 14 10:00:41 2022] GET /identity/v3/role_assignments?user.id=414b398a5bb04338a6ccd9b162381d4c&scope.system=all&role.id=73fb34f5dbe1404fa7ec015422951829 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:41.841063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef06358-1b5d-427e-81df-0267100b932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:41.841741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef06358-1b5d-427e-81df-0267100b932e None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:41.842018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef06358-1b5d-427e-81df-0267100b932e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:41.842288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef06358-1b5d-427e-81df-0267100b932e None admin] PATH_INFO: `/v3/groups` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:41.842982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef06358-1b5d-427e-81df-0267100b932e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:41.844428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef06358-1b5d-427e-81df-0267100b932e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:41.864142 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 160/310] 10.222.0.61 () {60 vars in 1134 bytes} [Thu Jul 14 10:00:41 2022] POST /identity/v3/groups => generated 229 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:00:42.294228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a2b88-26bc-467f-9255-d9d8cc0e751f None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:42.294759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a2b88-26bc-467f-9255-d9d8cc0e751f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:42.294759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a2b88-26bc-467f-9255-d9d8cc0e751f None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:42.296278 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 151/311] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:42.302177 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1d9ca-f954-413b-962e-18a36d497e05 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:42.302696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1d9ca-f954-413b-962e-18a36d497e05 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:42.303167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be1d9ca-f954-413b-962e-18a36d497e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:42.321492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1be1d9ca-f954-413b-962e-18a36d497e05 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:42.323200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1be1d9ca-f954-413b-962e-18a36d497e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:42.371569 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:42.398340 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 161/312] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:42.421310 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a2b88-26bc-467f-9255-d9d8cc0e751f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:42.486039 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:42.529638 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a2b88-26bc-467f-9255-d9d8cc0e751f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:42.565340 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d65ca-7cfe-4f72-a9d0-cc8970d6fc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:42.566337 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d65ca-7cfe-4f72-a9d0-cc8970d6fc3b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:42.566621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d65ca-7cfe-4f72-a9d0-cc8970d6fc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:42.566946 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d65ca-7cfe-4f72-a9d0-cc8970d6fc3b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:42.580206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d65ca-7cfe-4f72-a9d0-cc8970d6fc3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-6c7d65ca-7cfe-4f72-a9d0-cc8970d6fc3b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:42.588228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:42.589170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:42.589988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:42.589988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:42.589988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:42.589988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:42.589988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:42.589988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:42.589988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:42.589988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:42.589988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:42.589988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:42.589988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:42.589988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:42.589988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 152/313] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:42.596646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1d9ca-f954-413b-962e-18a36d497e05 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:42.700789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be1d9ca-f954-413b-962e-18a36d497e05 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:42.735806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed3e8c-c016-4b60-9ad6-86bfc5040a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:42.736486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed3e8c-c016-4b60-9ad6-86bfc5040a35 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:42.736770 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed3e8c-c016-4b60-9ad6-86bfc5040a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:42.737078 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ed3e8c-c016-4b60-9ad6-86bfc5040a35 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:42.737835 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed3e8c-c016-4b60-9ad6-86bfc5040a35 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:42.738144 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed3e8c-c016-4b60-9ad6-86bfc5040a35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:42.740268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ed3e8c-c016-4b60-9ad6-86bfc5040a35 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:42.748835 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 162/314] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:42.754033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d65ca-7cfe-4f72-a9d0-cc8970d6fc3b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:42.855149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d65ca-7cfe-4f72-a9d0-cc8970d6fc3b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:42.881283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:42.889478 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be6d63-62eb-4895-8775-c3ad229dcbd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:42.890817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be6d63-62eb-4895-8775-c3ad229dcbd3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:42.891353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be6d63-62eb-4895-8775-c3ad229dcbd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:42.891628 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04be6d63-62eb-4895-8775-c3ad229dcbd3 None admin] PATH_INFO: `/v3/users/a215c7f3d80941148fb04eb5b0fcc7c9` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:42.892845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be6d63-62eb-4895-8775-c3ad229dcbd3 None admin] RBAC: Authorizing `identity:get_user(user_id=a215c7f3d80941148fb04eb5b0fcc7c9)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:42.894322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04be6d63-62eb-4895-8775-c3ad229dcbd3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:42.896332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 153/315] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:42 2022] GET /identity/v3/users/a215c7f3d80941148fb04eb5b0fcc7c9 => generated 297 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:42.901851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed3e8c-c016-4b60-9ad6-86bfc5040a35 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:42.947407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:43.016807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ed3e8c-c016-4b60-9ad6-86bfc5040a35 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:43.054145 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fe4e7-dc7e-4ffc-9545-3e72919c6a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:43.054806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fe4e7-dc7e-4ffc-9545-3e72919c6a14 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:43.055088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fe4e7-dc7e-4ffc-9545-3e72919c6a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:43.055363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100fe4e7-dc7e-4ffc-9545-3e72919c6a14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:43.127830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fe4e7-dc7e-4ffc-9545-3e72919c6a14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:43.129238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100fe4e7-dc7e-4ffc-9545-3e72919c6a14 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:43.170916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:43.230360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 163/316] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:43.241118 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be6d63-62eb-4895-8775-c3ad229dcbd3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:43.348722 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04be6d63-62eb-4895-8775-c3ad229dcbd3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:43.384205 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc59d2d-1e00-4149-8021-a6dc2ead50cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:43.385558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc59d2d-1e00-4149-8021-a6dc2ead50cd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:43.386043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc59d2d-1e00-4149-8021-a6dc2ead50cd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:43.386550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc59d2d-1e00-4149-8021-a6dc2ead50cd None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:43.389689 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc59d2d-1e00-4149-8021-a6dc2ead50cd None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:43.391333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc59d2d-1e00-4149-8021-a6dc2ead50cd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:43.393631 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 154/317] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:43 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:43.399729 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fe4e7-dc7e-4ffc-9545-3e72919c6a14 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:43.498162 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738983f-dca4-467d-932d-b5cad4f326f0 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:43.498483 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738983f-dca4-467d-932d-b5cad4f326f0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:43.498706 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738983f-dca4-467d-932d-b5cad4f326f0 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:43.500324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 155/318] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:43.505113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e7ad6-41f2-4228-a14b-4faea3090456 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:43.505397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e7ad6-41f2-4228-a14b-4faea3090456 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:43.505671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e7ad6-41f2-4228-a14b-4faea3090456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:43.508024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100fe4e7-dc7e-4ffc-9545-3e72919c6a14 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:43.522878 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a26e7ad6-41f2-4228-a14b-4faea3090456 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:43.524066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a26e7ad6-41f2-4228-a14b-4faea3090456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:43.543395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63594c-ef85-4cc4-b136-9d1518521763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:43.544057 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63594c-ef85-4cc4-b136-9d1518521763 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:43.544337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63594c-ef85-4cc4-b136-9d1518521763 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:43.544602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63594c-ef85-4cc4-b136-9d1518521763 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:43.545480 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63594c-ef85-4cc4-b136-9d1518521763 None admin] RBAC: Adding query filter params (role.id=73fb34f5dbe1404fa7ec015422951829, scope.project.id=00a316c6194d45b392adff3ea7182a1c, user.id=a215c7f3d80941148fb04eb5b0fcc7c9) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:43.545796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63594c-ef85-4cc4-b136-9d1518521763 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:43.547238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63594c-ef85-4cc4-b136-9d1518521763 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:43.554151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 164/319] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:00:43 2022] GET /identity/v3/role_assignments?user.id=a215c7f3d80941148fb04eb5b0fcc7c9&scope.project.id=00a316c6194d45b392adff3ea7182a1c&role.id=73fb34f5dbe1404fa7ec015422951829 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:43.589450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:43.608713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 156/320] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:43.634997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63594c-ef85-4cc4-b136-9d1518521763 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:43.639004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:43.745804 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63594c-ef85-4cc4-b136-9d1518521763 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:43.779983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812f944-068f-4df6-abe5-f085f3328a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:43.780789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812f944-068f-4df6-abe5-f085f3328a1d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:43.781099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812f944-068f-4df6-abe5-f085f3328a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:43.781372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812f944-068f-4df6-abe5-f085f3328a1d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:43.792277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5812f944-068f-4df6-abe5-f085f3328a1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-5812f944-068f-4df6-abe5-f085f3328a1d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:43.799967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:43.800891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:43.802252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:43.802252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:43.802252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:43.802252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:43.802252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:43.802252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:43.802252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:43.802252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:43.802252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:43.802252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:43.802252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:43.802252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:43.802252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 165/321] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:43.807688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e7ad6-41f2-4228-a14b-4faea3090456 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:43.912445 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e7ad6-41f2-4228-a14b-4faea3090456 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:43.928321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79191234-639e-4d2c-844f-3747ccaf22ff None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:43.928616 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79191234-639e-4d2c-844f-3747ccaf22ff None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:43.928881 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79191234-639e-4d2c-844f-3747ccaf22ff None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:43.932854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 166/322] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:00:43.938196 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff39d6d-07b9-4838-beb2-16d08ddca550 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:43.938479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff39d6d-07b9-4838-beb2-16d08ddca550 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:43.938747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff39d6d-07b9-4838-beb2-16d08ddca550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:43.952626 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8f2d2-8b5c-4516-9734-4696d2c483da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:43.953329 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8f2d2-8b5c-4516-9734-4696d2c483da None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:43.953612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8f2d2-8b5c-4516-9734-4696d2c483da None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:43.953892 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba8f2d2-8b5c-4516-9734-4696d2c483da None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:43.954635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba8f2d2-8b5c-4516-9734-4696d2c483da None admin] RBAC: Adding query filter params (name=member) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:43.954938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba8f2d2-8b5c-4516-9734-4696d2c483da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:43.956300 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba8f2d2-8b5c-4516-9734-4696d2c483da None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:43.956683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5ff39d6d-07b9-4838-beb2-16d08ddca550 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:43.957948 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5ff39d6d-07b9-4838-beb2-16d08ddca550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:43.964975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 157/323] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:43.970387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8f2d2-8b5c-4516-9734-4696d2c483da None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:44.021676 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:44.029691 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:44.037821 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 167/324] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:44.042941 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c156c2-7dbb-4e59-8ec3-22ff357b2010 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:44.043229 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c156c2-7dbb-4e59-8ec3-22ff357b2010 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:44.043683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c156c2-7dbb-4e59-8ec3-22ff357b2010 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:44.045005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 168/325] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:44.050074 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f732380-290f-4876-b5e2-76043055df44 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:44.050365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f732380-290f-4876-b5e2-76043055df44 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:44.050645 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f732380-290f-4876-b5e2-76043055df44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:44.068223 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2f732380-290f-4876-b5e2-76043055df44 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:44.069442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2f732380-290f-4876-b5e2-76043055df44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:44.077147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba8f2d2-8b5c-4516-9734-4696d2c483da None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:44.110215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7ec88-d1c4-41cb-99bf-fc89f681e022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:44.111082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7ec88-d1c4-41cb-99bf-fc89f681e022 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:44.111355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7ec88-d1c4-41cb-99bf-fc89f681e022 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:44.111622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7ec88-d1c4-41cb-99bf-fc89f681e022 None admin] PATH_INFO: `/v3/users/1320dcaabb8a4ca29940cea8eb9dcc77` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:44.112811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7ec88-d1c4-41cb-99bf-fc89f681e022 None admin] RBAC: Authorizing `identity:get_user(user_id=1320dcaabb8a4ca29940cea8eb9dcc77)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:44.114851 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7ec88-d1c4-41cb-99bf-fc89f681e022 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:44.116621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 158/326] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:43 2022] GET /identity/v3/users/1320dcaabb8a4ca29940cea8eb9dcc77 => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:44.117248 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7ec88-d1c4-41cb-99bf-fc89f681e022 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:44.137075 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 169/327] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:44.138120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f732380-290f-4876-b5e2-76043055df44 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:44.224511 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7ec88-d1c4-41cb-99bf-fc89f681e022 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:44.233264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f732380-290f-4876-b5e2-76043055df44 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:44.243264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:44.257419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:44.262399 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3222d3-26f5-45a6-81a7-c766165af733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:44.263193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3222d3-26f5-45a6-81a7-c766165af733 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:44.263476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3222d3-26f5-45a6-81a7-c766165af733 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:44.263748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3222d3-26f5-45a6-81a7-c766165af733 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:44.267918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c231755-24e3-4975-8816-a66b0af6b8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:44.268570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c231755-24e3-4975-8816-a66b0af6b8fd None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:44.268845 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c231755-24e3-4975-8816-a66b0af6b8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:44.269152 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c231755-24e3-4975-8816-a66b0af6b8fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:44.270024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c231755-24e3-4975-8816-a66b0af6b8fd None admin] RBAC: Adding query filter params (role.id=3d9688bbbc15422cbfc59ded082fbe41, user.id=1320dcaabb8a4ca29940cea8eb9dcc77, scope.system=all) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:44.270331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c231755-24e3-4975-8816-a66b0af6b8fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:44.271750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c231755-24e3-4975-8816-a66b0af6b8fd None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:44.274180 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3222d3-26f5-45a6-81a7-c766165af733 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-bf3222d3-26f5-45a6-81a7-c766165af733 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:44.282238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:44.283117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:44.283912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:44.283912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:44.283912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:44.283912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:44.283912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:44.283912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:44.283912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:44.283912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:44.283912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:44.283912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:44.283912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:44.283912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:44.283912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 159/329] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:44.284574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3222d3-26f5-45a6-81a7-c766165af733 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:44.288602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 170/329] 10.222.0.61 () {56 vars in 1308 bytes} [Thu Jul 14 10:00:44 2022] GET /identity/v3/role_assignments?user.id=1320dcaabb8a4ca29940cea8eb9dcc77&scope.system=all&role.id=3d9688bbbc15422cbfc59ded082fbe41 => generated 510 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:44.289762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c231755-24e3-4975-8816-a66b0af6b8fd None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:44.398116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c231755-24e3-4975-8816-a66b0af6b8fd None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:44.411989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3222d3-26f5-45a6-81a7-c766165af733 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:44.440279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac0dbd-f1f0-4561-8601-b16be79925cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:44.440977 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac0dbd-f1f0-4561-8601-b16be79925cb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:44.441272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac0dbd-f1f0-4561-8601-b16be79925cb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:44.441590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ac0dbd-f1f0-4561-8601-b16be79925cb None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:44.442354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac0dbd-f1f0-4561-8601-b16be79925cb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:44.442664 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac0dbd-f1f0-4561-8601-b16be79925cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:44.444043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ac0dbd-f1f0-4561-8601-b16be79925cb None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:44.452797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 171/330] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:44.458596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ded60-f06c-4335-8fea-354d912c1a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:44.459551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ded60-f06c-4335-8fea-354d912c1a40 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:44.459653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac0dbd-f1f0-4561-8601-b16be79925cb None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:44.459728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ded60-f06c-4335-8fea-354d912c1a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:44.459956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89ded60-f06c-4335-8fea-354d912c1a40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:44.461243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ded60-f06c-4335-8fea-354d912c1a40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:44.462881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89ded60-f06c-4335-8fea-354d912c1a40 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:44.464472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 160/331] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:00:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:44.469868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ded60-f06c-4335-8fea-354d912c1a40 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:44.507572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:44.518878 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:44.560357 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ac0dbd-f1f0-4561-8601-b16be79925cb None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:44.573354 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89ded60-f06c-4335-8fea-354d912c1a40 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:44.593187 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2cb3c-88df-48f0-af39-4cd221ce0592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:44.594046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2cb3c-88df-48f0-af39-4cd221ce0592 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:44.594323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2cb3c-88df-48f0-af39-4cd221ce0592 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:44.594588 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2cb3c-88df-48f0-af39-4cd221ce0592 None admin] PATH_INFO: `/v3/users/414b398a5bb04338a6ccd9b162381d4c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:44.595759 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2cb3c-88df-48f0-af39-4cd221ce0592 None admin] RBAC: Authorizing `identity:get_user(user_id=414b398a5bb04338a6ccd9b162381d4c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:44.597267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2cb3c-88df-48f0-af39-4cd221ce0592 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:44.599510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 172/332] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:44 2022] GET /identity/v3/users/414b398a5bb04338a6ccd9b162381d4c => generated 293 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:44.604916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2cb3c-88df-48f0-af39-4cd221ce0592 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:44.606548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8329d06f-f74e-4e27-aee0-6df9128e7b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:44.607211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329d06f-f74e-4e27-aee0-6df9128e7b07 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:44.607490 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329d06f-f74e-4e27-aee0-6df9128e7b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:44.607763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8329d06f-f74e-4e27-aee0-6df9128e7b07 None admin] PATH_INFO: `/v3/groups` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:44.608447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8329d06f-f74e-4e27-aee0-6df9128e7b07 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:44.609921 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8329d06f-f74e-4e27-aee0-6df9128e7b07 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:44.624560 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 161/333] 10.222.0.61 () {60 vars in 1134 bytes} [Thu Jul 14 10:00:44 2022] POST /identity/v3/groups => generated 226 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:00:44.709719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2cb3c-88df-48f0-af39-4cd221ce0592 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:44.722931 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:44.744393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889ec8a-54fe-40d6-9f79-52bc23aa1044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:44.745297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889ec8a-54fe-40d6-9f79-52bc23aa1044 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:44.745571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889ec8a-54fe-40d6-9f79-52bc23aa1044 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:44.745838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889ec8a-54fe-40d6-9f79-52bc23aa1044 None admin] PATH_INFO: `/v3/system/users/414b398a5bb04338a6ccd9b162381d4c/roles/73fb34f5dbe1404fa7ec015422951829` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:44.747393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889ec8a-54fe-40d6-9f79-52bc23aa1044 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=414b398a5bb04338a6ccd9b162381d4c, role_id=73fb34f5dbe1404fa7ec015422951829)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:44.748770 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889ec8a-54fe-40d6-9f79-52bc23aa1044 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:44.757334 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 173/334] 10.222.0.61 () {58 vars in 1334 bytes} [Thu Jul 14 10:00:44 2022] PUT /identity/v3/system/users/414b398a5bb04338a6ccd9b162381d4c/roles/73fb34f5dbe1404fa7ec015422951829 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:00:46.995292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3444311-c553-4524-9530-7668dc5b8606 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:46.995801 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3444311-c553-4524-9530-7668dc5b8606 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:46.995877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3444311-c553-4524-9530-7668dc5b8606 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:46.997500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 162/335] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:47.003143 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3382b6-11c4-45c4-a50c-77ca2fa81620 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:47.003621 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3382b6-11c4-45c4-a50c-77ca2fa81620 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:47.003801 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3382b6-11c4-45c4-a50c-77ca2fa81620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:47.029266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-cd3382b6-11c4-45c4-a50c-77ca2fa81620 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:47.030625 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-cd3382b6-11c4-45c4-a50c-77ca2fa81620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:47.122956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 174/336] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:47.161209 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3444311-c553-4524-9530-7668dc5b8606 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:47.287692 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3444311-c553-4524-9530-7668dc5b8606 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:47.290265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:47.333842 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e145ac-d398-4668-94b9-1faf6438840a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:47.334645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e145ac-d398-4668-94b9-1faf6438840a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:47.334921 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e145ac-d398-4668-94b9-1faf6438840a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:47.335195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e145ac-d398-4668-94b9-1faf6438840a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:47.351369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e145ac-d398-4668-94b9-1faf6438840a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-63e145ac-d398-4668-94b9-1faf6438840a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:47.359385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:47.360249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:47.361075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:47.361075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:47.361075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:47.361075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:47.361075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:47.361075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:47.361075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:47.361075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:47.361075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:47.361075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:47.361075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 14 10:00:47.361075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:47.361075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 163/337] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:47.366965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3382b6-11c4-45c4-a50c-77ca2fa81620 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:47.450809 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fc2bc-2612-4d4f-9ec9-0d322f352074 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:47.451153 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fc2bc-2612-4d4f-9ec9-0d322f352074 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:47.451360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68fc2bc-2612-4d4f-9ec9-0d322f352074 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:47.453156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 164/338] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:47.461574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132ac10-69c5-483a-94d3-3a6579bebbe4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:47.461857 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132ac10-69c5-483a-94d3-3a6579bebbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:47.462132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132ac10-69c5-483a-94d3-3a6579bebbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:47.497106 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f132ac10-69c5-483a-94d3-3a6579bebbe4 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:47.498404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f132ac10-69c5-483a-94d3-3a6579bebbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:47.528233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3382b6-11c4-45c4-a50c-77ca2fa81620 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:47.536179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:47.580568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98bdb2-9396-4dbf-a8e0-da7ec0477e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:47.581316 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98bdb2-9396-4dbf-a8e0-da7ec0477e0f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:47.581596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98bdb2-9396-4dbf-a8e0-da7ec0477e0f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:47.581870 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98bdb2-9396-4dbf-a8e0-da7ec0477e0f None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:47.582637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98bdb2-9396-4dbf-a8e0-da7ec0477e0f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:47.582945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98bdb2-9396-4dbf-a8e0-da7ec0477e0f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:47.584321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98bdb2-9396-4dbf-a8e0-da7ec0477e0f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:47.601136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 175/339] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:47.606699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98bdb2-9396-4dbf-a8e0-da7ec0477e0f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:47.647741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 165/340] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:47.649125 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d39a9-d578-4db0-a2ca-6c53d2f70ada None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:47.649412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d39a9-d578-4db0-a2ca-6c53d2f70ada None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:47.649691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477d39a9-d578-4db0-a2ca-6c53d2f70ada None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:47.651325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 166/341] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:47.657327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e63087-964e-4017-9416-db0666a267cb None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:47.657607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e63087-964e-4017-9416-db0666a267cb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:47.657880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e63087-964e-4017-9416-db0666a267cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:47.686373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-61e63087-964e-4017-9416-db0666a267cb None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:47.687568 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-61e63087-964e-4017-9416-db0666a267cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:47.782257 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98bdb2-9396-4dbf-a8e0-da7ec0477e0f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:47.828673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81db486-86ec-49ac-9cd6-df80dcf85c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:47.829667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81db486-86ec-49ac-9cd6-df80dcf85c41 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:47.830105 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81db486-86ec-49ac-9cd6-df80dcf85c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:47.830633 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81db486-86ec-49ac-9cd6-df80dcf85c41 None admin] PATH_INFO: `/v3/users/414b398a5bb04338a6ccd9b162381d4c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:47.832352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81db486-86ec-49ac-9cd6-df80dcf85c41 None admin] RBAC: Authorizing `identity:get_user(user_id=414b398a5bb04338a6ccd9b162381d4c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:47.834058 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81db486-86ec-49ac-9cd6-df80dcf85c41 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:47.835792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 176/342] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:00:47 2022] GET /identity/v3/users/414b398a5bb04338a6ccd9b162381d4c => generated 293 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:47.837084 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81db486-86ec-49ac-9cd6-df80dcf85c41 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:47.843122 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 167/343] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:47.844302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb150ddc-b0f7-46c1-8373-7d5866b19a5c None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:47.845380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb150ddc-b0f7-46c1-8373-7d5866b19a5c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:47.845380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb150ddc-b0f7-46c1-8373-7d5866b19a5c None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:47.846910 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 168/344] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:47.847619 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb150ddc-b0f7-46c1-8373-7d5866b19a5c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:47.886216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:47.893491 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:47.974826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81db486-86ec-49ac-9cd6-df80dcf85c41 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:47.990955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb150ddc-b0f7-46c1-8373-7d5866b19a5c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:48.014770 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894e768-e8e4-4cdc-a332-5fc5c589e33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:48.015784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894e768-e8e4-4cdc-a332-5fc5c589e33f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:48.015890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894e768-e8e4-4cdc-a332-5fc5c589e33f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:48.016113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894e768-e8e4-4cdc-a332-5fc5c589e33f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:48.028399 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894e768-e8e4-4cdc-a332-5fc5c589e33f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-d894e768-e8e4-4cdc-a332-5fc5c589e33f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:48.036772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:48.037685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:48.038486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:48.038486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:48.038486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:48.038486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:48.038486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:48.038486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:48.038486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:48.038486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:48.038486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:48.038486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:48.038486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:48.038486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:48.038486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 177/345] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:48.039859 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf093a48-d408-4900-ae22-58356a2d5b7d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:48.039859 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf093a48-d408-4900-ae22-58356a2d5b7d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:48.039859 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf093a48-d408-4900-ae22-58356a2d5b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:48.046225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17defdca-4fed-4efc-9931-94de6ed6aa71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:48.046890 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17defdca-4fed-4efc-9931-94de6ed6aa71 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:48.049970 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17defdca-4fed-4efc-9931-94de6ed6aa71 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:48.050484 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17defdca-4fed-4efc-9931-94de6ed6aa71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:48.051553 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17defdca-4fed-4efc-9931-94de6ed6aa71 None admin] RBAC: Adding query filter params (role.id=73fb34f5dbe1404fa7ec015422951829, user.id=414b398a5bb04338a6ccd9b162381d4c, scope.system=all) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:48.052070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17defdca-4fed-4efc-9931-94de6ed6aa71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:48.053795 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17defdca-4fed-4efc-9931-94de6ed6aa71 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:48.065402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-cf093a48-d408-4900-ae22-58356a2d5b7d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:48.066622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-cf093a48-d408-4900-ae22-58356a2d5b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:48.067791 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 169/346] 10.222.0.61 () {56 vars in 1308 bytes} [Thu Jul 14 10:00:47 2022] GET /identity/v3/role_assignments?user.id=414b398a5bb04338a6ccd9b162381d4c&scope.system=all&role.id=73fb34f5dbe1404fa7ec015422951829 => generated 510 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:48.069131 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17defdca-4fed-4efc-9931-94de6ed6aa71 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:48.162811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:48.171404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 178/347] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:48.172299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf093a48-d408-4900-ae22-58356a2d5b7d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:48.185799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17defdca-4fed-4efc-9931-94de6ed6aa71 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:48.203307 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:48.241751 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b6247-9e48-45a0-b520-ba8ce29b6a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:48.242745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b6247-9e48-45a0-b520-ba8ce29b6a0c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:48.243129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b6247-9e48-45a0-b520-ba8ce29b6a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:48.243816 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3b6247-9e48-45a0-b520-ba8ce29b6a0c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:48.258950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3b6247-9e48-45a0-b520-ba8ce29b6a0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-2c3b6247-9e48-45a0-b520-ba8ce29b6a0c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:48.272822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:48.274023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:48.274871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:48.274871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:48.274871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:48.274871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:48.274871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:48.274871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:48.274871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:48.274871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:48.274871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:48.274871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:48.274871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:48.274871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:48.274871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 170/348] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:00:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:48.275533 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b6247-9e48-45a0-b520-ba8ce29b6a0c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:48.302213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf093a48-d408-4900-ae22-58356a2d5b7d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:48.339661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c244e-9661-4d09-96b9-603f21bd6249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:48.340655 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c244e-9661-4d09-96b9-603f21bd6249 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:48.341150 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c244e-9661-4d09-96b9-603f21bd6249 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:48.341600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475c244e-9661-4d09-96b9-603f21bd6249 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:48.342547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c244e-9661-4d09-96b9-603f21bd6249 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:48.343056 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c244e-9661-4d09-96b9-603f21bd6249 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:48.344647 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475c244e-9661-4d09-96b9-603f21bd6249 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:48.353955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 179/349] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:48.355262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c244e-9661-4d09-96b9-603f21bd6249 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:48.459234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3b6247-9e48-45a0-b520-ba8ce29b6a0c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:48.469334 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:48.481821 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475c244e-9661-4d09-96b9-603f21bd6249 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:48.516432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddc9a6-c1e3-4db3-9e77-8ac4f1e52626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:48.517712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddc9a6-c1e3-4db3-9e77-8ac4f1e52626 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:48.518049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddc9a6-c1e3-4db3-9e77-8ac4f1e52626 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:48.518324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bddc9a6-c1e3-4db3-9e77-8ac4f1e52626 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:48.531330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136964ef-1f63-426d-92dc-2c2e23d4fa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:48.532349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136964ef-1f63-426d-92dc-2c2e23d4fa1e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:48.532828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136964ef-1f63-426d-92dc-2c2e23d4fa1e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:48.533027 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bddc9a6-c1e3-4db3-9e77-8ac4f1e52626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:48.533594 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136964ef-1f63-426d-92dc-2c2e23d4fa1e None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:48.534607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136964ef-1f63-426d-92dc-2c2e23d4fa1e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:48.535110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136964ef-1f63-426d-92dc-2c2e23d4fa1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:48.537676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136964ef-1f63-426d-92dc-2c2e23d4fa1e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-9bddc9a6-c1e3-4db3-9e77-8ac4f1e52626 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:48.541727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:48.542621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:48.543421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:48.543421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:48.543421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:48.543421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:48.543421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:48.543421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:48.543421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:48.543421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:48.543421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:48.543421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:48.543421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:48.543421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:48.543421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 171/350] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:48.544073 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddc9a6-c1e3-4db3-9e77-8ac4f1e52626 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:48.549886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:48.551793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 180/351] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:00:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:48.553474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136964ef-1f63-426d-92dc-2c2e23d4fa1e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:48.681407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bddc9a6-c1e3-4db3-9e77-8ac4f1e52626 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:48.684761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136964ef-1f63-426d-92dc-2c2e23d4fa1e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:48.721743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448d00d-24fd-41e0-b5f2-3fe994a38312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:48.722071 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218c62ab-516f-4352-ac04-f6326d68d417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:48.722715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c62ab-516f-4352-ac04-f6326d68d417 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:48.722994 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c62ab-516f-4352-ac04-f6326d68d417 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:48.723264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218c62ab-516f-4352-ac04-f6326d68d417 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:48.723597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448d00d-24fd-41e0-b5f2-3fe994a38312 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:48.724065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448d00d-24fd-41e0-b5f2-3fe994a38312 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:48.724538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5448d00d-24fd-41e0-b5f2-3fe994a38312 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:48.725511 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448d00d-24fd-41e0-b5f2-3fe994a38312 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:48.726161 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448d00d-24fd-41e0-b5f2-3fe994a38312 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:48.728453 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5448d00d-24fd-41e0-b5f2-3fe994a38312 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:48.743428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 181/352] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:48.745880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448d00d-24fd-41e0-b5f2-3fe994a38312 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:48.765484 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:48.820918 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c62ab-516f-4352-ac04-f6326d68d417 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:48.822425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218c62ab-516f-4352-ac04-f6326d68d417 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:48.824393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:48.886923 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5448d00d-24fd-41e0-b5f2-3fe994a38312 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:48.927149 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991b525-05f2-473a-8ea0-3586bc0caac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:48.927806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991b525-05f2-473a-8ea0-3586bc0caac4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:48.928083 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991b525-05f2-473a-8ea0-3586bc0caac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:48.928380 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991b525-05f2-473a-8ea0-3586bc0caac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:48.952162 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 172/353] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:48.953377 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42119ebf-d9da-45c9-b0b7-3b9f60600d44 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:48.953663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42119ebf-d9da-45c9-b0b7-3b9f60600d44 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:48.954384 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42119ebf-d9da-45c9-b0b7-3b9f60600d44 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:48.956657 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 173/354] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:48.960652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42119ebf-d9da-45c9-b0b7-3b9f60600d44 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:49.010910 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991b525-05f2-473a-8ea0-3586bc0caac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:49.013062 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0991b525-05f2-473a-8ea0-3586bc0caac4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:49.029022 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:49.089741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42119ebf-d9da-45c9-b0b7-3b9f60600d44 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:49.091391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:49.128102 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 182/355] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:49.128423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f48d0-bb58-4312-9262-9826d6cc3a35 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:49.128710 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f48d0-bb58-4312-9262-9826d6cc3a35 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:49.129022 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72f48d0-bb58-4312-9262-9826d6cc3a35 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:49.130451 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 183/356] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:49.131081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f48d0-bb58-4312-9262-9826d6cc3a35 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:49.137780 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94c721-bc1f-42a9-90ec-babf2db195bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:49.138725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94c721-bc1f-42a9-90ec-babf2db195bd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:49.139201 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94c721-bc1f-42a9-90ec-babf2db195bd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:49.139476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a94c721-bc1f-42a9-90ec-babf2db195bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:49.223274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94c721-bc1f-42a9-90ec-babf2db195bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:49.224731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a94c721-bc1f-42a9-90ec-babf2db195bd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:49.248499 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72f48d0-bb58-4312-9262-9826d6cc3a35 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:49.263677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:49.290219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e70ad2-7d8c-47b8-80ad-a49c6efb7238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:49.291086 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e70ad2-7d8c-47b8-80ad-a49c6efb7238 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:49.291366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e70ad2-7d8c-47b8-80ad-a49c6efb7238 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:49.291637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e70ad2-7d8c-47b8-80ad-a49c6efb7238 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:49.292818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e70ad2-7d8c-47b8-80ad-a49c6efb7238 None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:49.294274 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e70ad2-7d8c-47b8-80ad-a49c6efb7238 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:49.295977 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 184/357] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:49 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:49.298135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89117ed-4667-4c8b-b828-dae9ce367818 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:49.298425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89117ed-4667-4c8b-b828-dae9ce367818 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:49.298698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89117ed-4667-4c8b-b828-dae9ce367818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:49.319119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f89117ed-4667-4c8b-b828-dae9ce367818 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:49.320317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f89117ed-4667-4c8b-b828-dae9ce367818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:49.349475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 174/358] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:49.350095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94c721-bc1f-42a9-90ec-babf2db195bd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:49.352461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:49.394480 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 185/359] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:49.395768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438dfe5-4fd3-4ecf-92f6-7607f95fa89a None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:49.396052 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438dfe5-4fd3-4ecf-92f6-7607f95fa89a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:49.396328 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5438dfe5-4fd3-4ecf-92f6-7607f95fa89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:49.414426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5438dfe5-4fd3-4ecf-92f6-7607f95fa89a None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:49.415602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5438dfe5-4fd3-4ecf-92f6-7607f95fa89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:49.471255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a94c721-bc1f-42a9-90ec-babf2db195bd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:49.505110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 186/360] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:49.505856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438dfe5-4fd3-4ecf-92f6-7607f95fa89a None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:49.519173 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6a881-74e2-425b-bef9-239845538461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:49.521091 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6a881-74e2-425b-bef9-239845538461 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:49.521776 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6a881-74e2-425b-bef9-239845538461 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:49.522275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b6a881-74e2-425b-bef9-239845538461 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:49.525530 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6a881-74e2-425b-bef9-239845538461 None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:49.528614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b6a881-74e2-425b-bef9-239845538461 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:49.532514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 175/361] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:49 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:49.535765 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6a881-74e2-425b-bef9-239845538461 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:49.555817 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:49.630987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5438dfe5-4fd3-4ecf-92f6-7607f95fa89a None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:49.633541 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:49.671265 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf360b-3998-407b-a9b5-f2dcf76a3930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:49.672374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf360b-3998-407b-a9b5-f2dcf76a3930 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:49.672662 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf360b-3998-407b-a9b5-f2dcf76a3930 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:49.672954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbf360b-3998-407b-a9b5-f2dcf76a3930 None admin] PATH_INFO: `/v3/groups/4c15fded3d4f4a50a54b7eab156e431e` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:49.674560 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b6a881-74e2-425b-bef9-239845538461 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:49.680782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf360b-3998-407b-a9b5-f2dcf76a3930 None admin] RBAC: Authorizing `identity:get_group(group_id=4c15fded3d4f4a50a54b7eab156e431e)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:49.682325 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbf360b-3998-407b-a9b5-f2dcf76a3930 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:49.684038 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 187/362] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Jul 14 10:00:49 2022] GET /identity/v3/groups/4c15fded3d4f4a50a54b7eab156e431e => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:49.685953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf360b-3998-407b-a9b5-f2dcf76a3930 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:49.718243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfde79-0124-461d-8efb-ab8c9c1cf8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:49.719110 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfde79-0124-461d-8efb-ab8c9c1cf8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:49.719390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfde79-0124-461d-8efb-ab8c9c1cf8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:49.719655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bfde79-0124-461d-8efb-ab8c9c1cf8f2 None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:49.721540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfde79-0124-461d-8efb-ab8c9c1cf8f2 None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:49.722995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bfde79-0124-461d-8efb-ab8c9c1cf8f2 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:49.724881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 176/363] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:49 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:49.725501 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfde79-0124-461d-8efb-ab8c9c1cf8f2 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:49.808115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbf360b-3998-407b-a9b5-f2dcf76a3930 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:49.812043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:49.846147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bfde79-0124-461d-8efb-ab8c9c1cf8f2 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:49.849306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdada55-4680-48df-b839-73102af3815f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:49.850126 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdada55-4680-48df-b839-73102af3815f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:49.850410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdada55-4680-48df-b839-73102af3815f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:49.850682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdada55-4680-48df-b839-73102af3815f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:49.863990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdada55-4680-48df-b839-73102af3815f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-fcdada55-4680-48df-b839-73102af3815f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:49.872808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:49.873779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:49.874590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:49.874590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:49.874590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:49.874590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:49.874590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:49.874590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:49.874590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:49.874590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:49.874590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:49.874590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:49.874590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:49.874590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:49.875225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 188/364] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:00:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:49.877885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdada55-4680-48df-b839-73102af3815f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:49.888376 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58415bc1-dbcd-4c72-b38b-b7fd79ccb8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:49.889219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58415bc1-dbcd-4c72-b38b-b7fd79ccb8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:49.889485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58415bc1-dbcd-4c72-b38b-b7fd79ccb8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:49.889799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58415bc1-dbcd-4c72-b38b-b7fd79ccb8d2 None admin] PATH_INFO: `/v3/groups/4c15fded3d4f4a50a54b7eab156e431e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:49.892812 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58415bc1-dbcd-4c72-b38b-b7fd79ccb8d2 None admin] RBAC: Authorizing `identity:get_group(group_id=4c15fded3d4f4a50a54b7eab156e431e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:49.892812 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58415bc1-dbcd-4c72-b38b-b7fd79ccb8d2 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:49.894258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 177/365] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Jul 14 10:00:49 2022] GET /identity/v3/groups/4c15fded3d4f4a50a54b7eab156e431e => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:49.895025 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58415bc1-dbcd-4c72-b38b-b7fd79ccb8d2 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:49.904237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:49.996335 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdada55-4680-48df-b839-73102af3815f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:50.024511 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58415bc1-dbcd-4c72-b38b-b7fd79ccb8d2 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:50.036773 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c266b5c-13b9-4fa7-b0fe-950fd88aeae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:50.039584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c266b5c-13b9-4fa7-b0fe-950fd88aeae1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:50.039866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c266b5c-13b9-4fa7-b0fe-950fd88aeae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:50.040146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c266b5c-13b9-4fa7-b0fe-950fd88aeae1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:50.052771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c266b5c-13b9-4fa7-b0fe-950fd88aeae1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-5c266b5c-13b9-4fa7-b0fe-950fd88aeae1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:50.060820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:50.061750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:50.062689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:50.062689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:50.062689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:50.062689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:50.062689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:50.062689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:50.062689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:50.062689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:50.062689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:50.062689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:50.062689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:50.062689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:50.062689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 189/366] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:00:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:50.063977 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c266b5c-13b9-4fa7-b0fe-950fd88aeae1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:50.064906 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6db33-fea7-47c7-83be-8b434c35a5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:50.065598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6db33-fea7-47c7-83be-8b434c35a5be None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:50.065877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6db33-fea7-47c7-83be-8b434c35a5be None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:50.066152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab6db33-fea7-47c7-83be-8b434c35a5be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:50.067127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6db33-fea7-47c7-83be-8b434c35a5be None admin] RBAC: Adding query filter params (group.id=4c15fded3d4f4a50a54b7eab156e431e, role.id=3d9688bbbc15422cbfc59ded082fbe41, scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:50.068574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6db33-fea7-47c7-83be-8b434c35a5be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:50.070075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab6db33-fea7-47c7-83be-8b434c35a5be None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:50.075970 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 178/367] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Jul 14 10:00:49 2022] GET /identity/v3/role_assignments?group.id=4c15fded3d4f4a50a54b7eab156e431e&scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c&role.id=3d9688bbbc15422cbfc59ded082fbe41 => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:50.076859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6db33-fea7-47c7-83be-8b434c35a5be None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:50.092649 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:50.185999 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:50.212634 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c266b5c-13b9-4fa7-b0fe-950fd88aeae1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:50.230307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab6db33-fea7-47c7-83be-8b434c35a5be None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:50.267453 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541f57d-03eb-47bc-ab55-437a302b44c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:50.268324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541f57d-03eb-47bc-ab55-437a302b44c1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:50.268604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541f57d-03eb-47bc-ab55-437a302b44c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:50.268875 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b541f57d-03eb-47bc-ab55-437a302b44c1 None admin] PATH_INFO: `/v3/groups/4c15fded3d4f4a50a54b7eab156e431e` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:50.270103 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541f57d-03eb-47bc-ab55-437a302b44c1 None admin] RBAC: Authorizing `identity:get_group(group_id=4c15fded3d4f4a50a54b7eab156e431e)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:50.271542 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b541f57d-03eb-47bc-ab55-437a302b44c1 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:50.274502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 190/368] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Jul 14 10:00:50 2022] GET /identity/v3/groups/4c15fded3d4f4a50a54b7eab156e431e => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:50.274502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541f57d-03eb-47bc-ab55-437a302b44c1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:50.283449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b2373-b94d-4a99-ac07-c99752d653af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:50.284129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b2373-b94d-4a99-ac07-c99752d653af None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:50.284408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b2373-b94d-4a99-ac07-c99752d653af None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:50.284678 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b2373-b94d-4a99-ac07-c99752d653af None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:50.285461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b2373-b94d-4a99-ac07-c99752d653af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:50.285767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b2373-b94d-4a99-ac07-c99752d653af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:50.287207 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b2373-b94d-4a99-ac07-c99752d653af None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:50.296238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 179/369] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:00:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:50.297113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b2373-b94d-4a99-ac07-c99752d653af None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:50.380017 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:50.390100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b541f57d-03eb-47bc-ab55-437a302b44c1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:50.407499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b2373-b94d-4a99-ac07-c99752d653af None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:50.426363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cc2d1-d329-4ed9-acbd-0e77362256ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:50.427027 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cc2d1-d329-4ed9-acbd-0e77362256ea None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:50.427310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cc2d1-d329-4ed9-acbd-0e77362256ea None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:50.427584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01cc2d1-d329-4ed9-acbd-0e77362256ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:50.428433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01cc2d1-d329-4ed9-acbd-0e77362256ea None admin] RBAC: Adding query filter params (group.id=4c15fded3d4f4a50a54b7eab156e431e, role.id=f4e1d6f34f674f539ea7f2564646c0a5, scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:50.428746 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01cc2d1-d329-4ed9-acbd-0e77362256ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:50.430212 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01cc2d1-d329-4ed9-acbd-0e77362256ea None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:50.436388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 191/370] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Jul 14 10:00:50 2022] GET /identity/v3/role_assignments?group.id=4c15fded3d4f4a50a54b7eab156e431e&scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c&role.id=f4e1d6f34f674f539ea7f2564646c0a5 => generated 260 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:50.437980 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:50.439868 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cc2d1-d329-4ed9-acbd-0e77362256ea None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:50.443960 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6841c70-6e34-4e15-8e4d-2284557861dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:50.444688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6841c70-6e34-4e15-8e4d-2284557861dd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:50.445079 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6841c70-6e34-4e15-8e4d-2284557861dd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:50.445449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6841c70-6e34-4e15-8e4d-2284557861dd None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:50.446558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6841c70-6e34-4e15-8e4d-2284557861dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:50.446675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6841c70-6e34-4e15-8e4d-2284557861dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:50.448093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6841c70-6e34-4e15-8e4d-2284557861dd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:50.464141 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 180/371] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:50.465276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6841c70-6e34-4e15-8e4d-2284557861dd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:50.572767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01cc2d1-d329-4ed9-acbd-0e77362256ea None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:50.593428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6841c70-6e34-4e15-8e4d-2284557861dd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:50.624676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50a238-ced6-44b2-af1c-9fa4dbc1b976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:50.625668 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50a238-ced6-44b2-af1c-9fa4dbc1b976 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:50.626016 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50a238-ced6-44b2-af1c-9fa4dbc1b976 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:50.627216 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50a238-ced6-44b2-af1c-9fa4dbc1b976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:50.628352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50a238-ced6-44b2-af1c-9fa4dbc1b976 None admin] RBAC: Adding query filter params (group.id=4c15fded3d4f4a50a54b7eab156e431e, role.id=3d9688bbbc15422cbfc59ded082fbe41, scope.project.id=00a316c6194d45b392adff3ea7182a1c) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:50.628671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50a238-ced6-44b2-af1c-9fa4dbc1b976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:50.630161 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50a238-ced6-44b2-af1c-9fa4dbc1b976 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:50.631671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a4db4-c0d8-4a9c-8682-d7e03c7e174d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:50.632410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a4db4-c0d8-4a9c-8682-d7e03c7e174d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:50.632782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a4db4-c0d8-4a9c-8682-d7e03c7e174d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:50.633170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a4db4-c0d8-4a9c-8682-d7e03c7e174d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:50.637090 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 192/372] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Jul 14 10:00:50 2022] GET /identity/v3/role_assignments?group.id=4c15fded3d4f4a50a54b7eab156e431e&scope.project.id=00a316c6194d45b392adff3ea7182a1c&role.id=3d9688bbbc15422cbfc59ded082fbe41 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:50.637931 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50a238-ced6-44b2-af1c-9fa4dbc1b976 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:50.683679 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:50.759284 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:50.773102 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a4db4-c0d8-4a9c-8682-d7e03c7e174d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:50.774604 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a4db4-c0d8-4a9c-8682-d7e03c7e174d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:50.834419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50a238-ced6-44b2-af1c-9fa4dbc1b976 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:50.896641 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eec295-9a9d-423b-bc2d-2a5a52918f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:50.897326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eec295-9a9d-423b-bc2d-2a5a52918f4f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:50.897611 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eec295-9a9d-423b-bc2d-2a5a52918f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:50.897887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eec295-9a9d-423b-bc2d-2a5a52918f4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:50.963360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 181/373] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:50.964286 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a4db4-c0d8-4a9c-8682-d7e03c7e174d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:51.017266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eec295-9a9d-423b-bc2d-2a5a52918f4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:51.018656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eec295-9a9d-423b-bc2d-2a5a52918f4f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:51.058141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:51.128943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:51.146291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a4db4-c0d8-4a9c-8682-d7e03c7e174d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:51.193497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 193/374] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:51.199091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eec295-9a9d-423b-bc2d-2a5a52918f4f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:51.215274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6f709-57c3-4994-afac-1024fd51a377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:51.216171 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6f709-57c3-4994-afac-1024fd51a377 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:51.216458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6f709-57c3-4994-afac-1024fd51a377 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:51.217092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6f709-57c3-4994-afac-1024fd51a377 None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:51.219089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6f709-57c3-4994-afac-1024fd51a377 None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:51.221552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6f709-57c3-4994-afac-1024fd51a377 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:51.224258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 182/375] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:50 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:51.229736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6f709-57c3-4994-afac-1024fd51a377 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:51.341945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eec295-9a9d-423b-bc2d-2a5a52918f4f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:51.363376 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:51.382398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa333f-7ddf-403f-89b7-f18c1bcf2dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:51.385458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa333f-7ddf-403f-89b7-f18c1bcf2dd7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:51.386139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa333f-7ddf-403f-89b7-f18c1bcf2dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:51.388182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfa333f-7ddf-403f-89b7-f18c1bcf2dd7 None admin] PATH_INFO: `/v3/projects/4677fbdc96dd49fca8554ea63b0d5a7c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:51.390707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa333f-7ddf-403f-89b7-f18c1bcf2dd7 None admin] RBAC: Authorizing `identity:get_project(project_id=4677fbdc96dd49fca8554ea63b0d5a7c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:51.393083 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfa333f-7ddf-403f-89b7-f18c1bcf2dd7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:51.395196 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 194/376] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:51 2022] GET /identity/v3/projects/4677fbdc96dd49fca8554ea63b0d5a7c => generated 344 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:51.403099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa333f-7ddf-403f-89b7-f18c1bcf2dd7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:51.444954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6f709-57c3-4994-afac-1024fd51a377 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:51.478265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565a2d1-e72f-4042-a5d4-d7caab932f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:51.479132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565a2d1-e72f-4042-a5d4-d7caab932f3f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:51.479413 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565a2d1-e72f-4042-a5d4-d7caab932f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:51.479686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7565a2d1-e72f-4042-a5d4-d7caab932f3f None admin] PATH_INFO: `/v3/groups/4c15fded3d4f4a50a54b7eab156e431e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:51.480902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565a2d1-e72f-4042-a5d4-d7caab932f3f None admin] RBAC: Authorizing `identity:get_group(group_id=4c15fded3d4f4a50a54b7eab156e431e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:51.482325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7565a2d1-e72f-4042-a5d4-d7caab932f3f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:51.488704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 183/377] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Jul 14 10:00:51 2022] GET /identity/v3/groups/4c15fded3d4f4a50a54b7eab156e431e => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:51.489785 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565a2d1-e72f-4042-a5d4-d7caab932f3f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:51.551681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:51.608306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfa333f-7ddf-403f-89b7-f18c1bcf2dd7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:51.653142 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7565a2d1-e72f-4042-a5d4-d7caab932f3f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:51.689808 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c1553-f8b3-4149-9e27-9739dcf5cd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:51.690811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c1553-f8b3-4149-9e27-9739dcf5cd68 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:51.691243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c1553-f8b3-4149-9e27-9739dcf5cd68 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:51.691671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c1553-f8b3-4149-9e27-9739dcf5cd68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:51.692791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4d7c16-2966-48b1-8c50-16a189d7edba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:51.693240 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c1553-f8b3-4149-9e27-9739dcf5cd68 None admin] RBAC: Adding query filter params (group.id=4c15fded3d4f4a50a54b7eab156e431e, role.id=f4e1d6f34f674f539ea7f2564646c0a5, scope.project.id=00a316c6194d45b392adff3ea7182a1c) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:51.693711 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d7c16-2966-48b1-8c50-16a189d7edba None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:51.693849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c1553-f8b3-4149-9e27-9739dcf5cd68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:51.693993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d7c16-2966-48b1-8c50-16a189d7edba None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:51.694273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4d7c16-2966-48b1-8c50-16a189d7edba None admin] PATH_INFO: `/v3/groups/75718cd66c97465daa0dfd23189109d5` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:51.695880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c1553-f8b3-4149-9e27-9739dcf5cd68 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:51.703304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 184/378] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Jul 14 10:00:51 2022] GET /identity/v3/role_assignments?group.id=4c15fded3d4f4a50a54b7eab156e431e&scope.project.id=00a316c6194d45b392adff3ea7182a1c&role.id=f4e1d6f34f674f539ea7f2564646c0a5 => generated 260 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:51.706440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d7c16-2966-48b1-8c50-16a189d7edba None admin] RBAC: Authorizing `identity:get_group(group_id=75718cd66c97465daa0dfd23189109d5)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:51.708076 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4d7c16-2966-48b1-8c50-16a189d7edba None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:51.710465 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 195/379] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Jul 14 10:00:51 2022] GET /identity/v3/groups/75718cd66c97465daa0dfd23189109d5 => generated 229 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:51.716574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c1553-f8b3-4149-9e27-9739dcf5cd68 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:51.818796 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:51.825957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c1553-f8b3-4149-9e27-9739dcf5cd68 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:51.863370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e5206-7c32-4336-99d0-696cd9940fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:51.864373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e5206-7c32-4336-99d0-696cd9940fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:51.864813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e5206-7c32-4336-99d0-696cd9940fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:51.865249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e5206-7c32-4336-99d0-696cd9940fb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:51.866270 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e5206-7c32-4336-99d0-696cd9940fb9 None admin] RBAC: Adding query filter params (group.id=75718cd66c97465daa0dfd23189109d5, role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=4677fbdc96dd49fca8554ea63b0d5a7c) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:51.866743 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e5206-7c32-4336-99d0-696cd9940fb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:51.868370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e5206-7c32-4336-99d0-696cd9940fb9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:51.875138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 185/380] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Jul 14 10:00:51 2022] GET /identity/v3/role_assignments?group.id=75718cd66c97465daa0dfd23189109d5&scope.project.id=4677fbdc96dd49fca8554ea63b0d5a7c&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:52.312467 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a375cb-7e91-43c3-8c03-c3f58ff95068 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:52.313028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a375cb-7e91-43c3-8c03-c3f58ff95068 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:52.313028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a375cb-7e91-43c3-8c03-c3f58ff95068 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:52.315282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 196/381] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:52.320955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84e34c-1ddc-45e1-b254-1e215de4e95f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:52.321352 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84e34c-1ddc-45e1-b254-1e215de4e95f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:52.321505 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84e34c-1ddc-45e1-b254-1e215de4e95f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:52.338191 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8b84e34c-1ddc-45e1-b254-1e215de4e95f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:52.339371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8b84e34c-1ddc-45e1-b254-1e215de4e95f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:52.416731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 186/382] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:52.462392 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a375cb-7e91-43c3-8c03-c3f58ff95068 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:52.536522 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:52.590342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a375cb-7e91-43c3-8c03-c3f58ff95068 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:52.637653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebfb916-f149-45f0-afc0-7b9017ae9f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:52.638452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfb916-f149-45f0-afc0-7b9017ae9f8f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:52.638729 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfb916-f149-45f0-afc0-7b9017ae9f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:52.638996 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebfb916-f149-45f0-afc0-7b9017ae9f8f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:52.655117 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebfb916-f149-45f0-afc0-7b9017ae9f8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-eebfb916-f149-45f0-afc0-7b9017ae9f8f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:52.662799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:52.663749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:52.664607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:52.664607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:52.664607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:52.664607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:52.664607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:52.664607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:52.664607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:52.664607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:52.664607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:52.664607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:52.664607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:52.664607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:52.669104 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 197/383] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:52.670448 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84e34c-1ddc-45e1-b254-1e215de4e95f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:52.782095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:52.797017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84e34c-1ddc-45e1-b254-1e215de4e95f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:52.842650 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe0372-5da1-44d2-8f49-f6d7952556af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:52.843323 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe0372-5da1-44d2-8f49-f6d7952556af None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:52.843602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe0372-5da1-44d2-8f49-f6d7952556af None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:52.843873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe0372-5da1-44d2-8f49-f6d7952556af None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:52.844665 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe0372-5da1-44d2-8f49-f6d7952556af None admin] RBAC: Adding query filter params (name=member) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:52.844984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe0372-5da1-44d2-8f49-f6d7952556af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:52.846342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe0372-5da1-44d2-8f49-f6d7952556af None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:52.851293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad281d1-6525-4279-8d48-ed0e6bde478d None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:52.851579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad281d1-6525-4279-8d48-ed0e6bde478d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:52.851853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad281d1-6525-4279-8d48-ed0e6bde478d None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:52.853309 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 198/384] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:00:52.862084 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a514882-9b63-4bd2-aee9-607ecedfd87e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:52.862366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a514882-9b63-4bd2-aee9-607ecedfd87e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:52.862638 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a514882-9b63-4bd2-aee9-607ecedfd87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:52.876986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 187/385] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:52.878226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe0372-5da1-44d2-8f49-f6d7952556af None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:52.889935 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3a514882-9b63-4bd2-aee9-607ecedfd87e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:52.891266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3a514882-9b63-4bd2-aee9-607ecedfd87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:53.011333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 199/386] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:53.018742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe0372-5da1-44d2-8f49-f6d7952556af None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:53.066163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1942a1-dabd-47c3-9e3d-612539d2810f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:53.067501 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1942a1-dabd-47c3-9e3d-612539d2810f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:53.067919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1942a1-dabd-47c3-9e3d-612539d2810f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:53.068342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1942a1-dabd-47c3-9e3d-612539d2810f None admin] PATH_INFO: `/v3/groups/4c15fded3d4f4a50a54b7eab156e431e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:53.069740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1942a1-dabd-47c3-9e3d-612539d2810f None admin] RBAC: Authorizing `identity:get_group(group_id=4c15fded3d4f4a50a54b7eab156e431e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:53.071352 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1942a1-dabd-47c3-9e3d-612539d2810f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:53.074579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 188/387] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Jul 14 10:00:52 2022] GET /identity/v3/groups/4c15fded3d4f4a50a54b7eab156e431e => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:53.080356 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a514882-9b63-4bd2-aee9-607ecedfd87e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:53.080530 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1942a1-dabd-47c3-9e3d-612539d2810f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:53.118010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:53.134372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:53.197916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a514882-9b63-4bd2-aee9-607ecedfd87e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:53.234552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f762d-4aab-46ba-a116-f5bde2600f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:53.236430 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f762d-4aab-46ba-a116-f5bde2600f92 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:53.236960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f762d-4aab-46ba-a116-f5bde2600f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:53.237443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f762d-4aab-46ba-a116-f5bde2600f92 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:53.239033 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f762d-4aab-46ba-a116-f5bde2600f92 None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:53.240680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f762d-4aab-46ba-a116-f5bde2600f92 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:53.242621 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 200/388] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:53 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:53.248775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f7e37-9c2e-4b05-830c-9b9f8eefb191 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:53.249027 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1942a1-dabd-47c3-9e3d-612539d2810f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:53.249460 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f7e37-9c2e-4b05-830c-9b9f8eefb191 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:53.249913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f7e37-9c2e-4b05-830c-9b9f8eefb191 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:53.252763 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 201/389] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:53.253918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f7e37-9c2e-4b05-830c-9b9f8eefb191 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:53.286737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f90ff8-9aae-47f3-b2b5-47d087fb963a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:53.287569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f90ff8-9aae-47f3-b2b5-47d087fb963a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:53.287848 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f90ff8-9aae-47f3-b2b5-47d087fb963a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:53.288119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f90ff8-9aae-47f3-b2b5-47d087fb963a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:53.302972 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f90ff8-9aae-47f3-b2b5-47d087fb963a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-52f90ff8-9aae-47f3-b2b5-47d087fb963a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:53.312593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:53.313544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:53.314773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:53.314773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:53.314773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:53.314773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:53.314773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:53.314773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:53.314773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:53.314773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:53.314773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:53.314773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:53.314773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:53.314773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:53.314773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 189/390] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:00:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:00:53.315539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d1ce7-2338-49ba-8f30-dd8eda135668 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:53.315831 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d1ce7-2338-49ba-8f30-dd8eda135668 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:53.315982 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5d1ce7-2338-49ba-8f30-dd8eda135668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:53.337762 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ac5d1ce7-2338-49ba-8f30-dd8eda135668 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:53.338879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ac5d1ce7-2338-49ba-8f30-dd8eda135668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:53.388478 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f7e37-9c2e-4b05-830c-9b9f8eefb191 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:53.401562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:53.422291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 190/391] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:53.424272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d1ce7-2338-49ba-8f30-dd8eda135668 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:53.428809 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d43bdc-034a-4b7f-b2b9-b40f6058c550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:53.429810 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d43bdc-034a-4b7f-b2b9-b40f6058c550 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:53.430088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d43bdc-034a-4b7f-b2b9-b40f6058c550 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:53.430368 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d43bdc-034a-4b7f-b2b9-b40f6058c550 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c/groups/4c15fded3d4f4a50a54b7eab156e431e/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:53.432457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d43bdc-034a-4b7f-b2b9-b40f6058c550 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee948cfa1f0c4c65a69fc5204872db0c, group_id=4c15fded3d4f4a50a54b7eab156e431e, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:53.433926 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d43bdc-034a-4b7f-b2b9-b40f6058c550 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:53.436622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:53.456198 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 202/392] 10.222.0.61 () {58 vars in 1442 bytes} [Thu Jul 14 10:00:53 2022] PUT /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c/groups/4c15fded3d4f4a50a54b7eab156e431e/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:00:53.457788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d43bdc-034a-4b7f-b2b9-b40f6058c550 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:53.612310 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5d1ce7-2338-49ba-8f30-dd8eda135668 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:53.656372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d43bdc-034a-4b7f-b2b9-b40f6058c550 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:53.657168 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54770173-1af8-4ffc-83d3-df7e8f2c1f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:53.657673 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54770173-1af8-4ffc-83d3-df7e8f2c1f72 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:53.657955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54770173-1af8-4ffc-83d3-df7e8f2c1f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:53.658231 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54770173-1af8-4ffc-83d3-df7e8f2c1f72 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:53.658988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54770173-1af8-4ffc-83d3-df7e8f2c1f72 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:53.659294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54770173-1af8-4ffc-83d3-df7e8f2c1f72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:53.660862 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54770173-1af8-4ffc-83d3-df7e8f2c1f72 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:53.671968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 191/393] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:00:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:53.677447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54770173-1af8-4ffc-83d3-df7e8f2c1f72 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:53.705845 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1281565-3f4b-460c-a02e-a4de47dc354f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:53.706665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1281565-3f4b-460c-a02e-a4de47dc354f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:53.706945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1281565-3f4b-460c-a02e-a4de47dc354f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:53.707220 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1281565-3f4b-460c-a02e-a4de47dc354f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:53.721591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1281565-3f4b-460c-a02e-a4de47dc354f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-b1281565-3f4b-460c-a02e-a4de47dc354f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:53.730956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:53.731866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:53.737653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:53.737653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:53.737653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:53.737653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:53.737653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:53.737653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:53.737653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:53.737653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:53.737653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:53.737653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:53.737653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:53.737653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:53.737653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 203/394] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:00:53.738651 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1281565-3f4b-460c-a02e-a4de47dc354f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:53.773398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:53.810679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:53.852707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1281565-3f4b-460c-a02e-a4de47dc354f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:53.890791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1373d-6c85-4730-8306-f5d7e7891723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:53.891391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1373d-6c85-4730-8306-f5d7e7891723 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:53.891675 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1373d-6c85-4730-8306-f5d7e7891723 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:53.891947 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1373d-6c85-4730-8306-f5d7e7891723 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:53.892714 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1373d-6c85-4730-8306-f5d7e7891723 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:53.893067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1373d-6c85-4730-8306-f5d7e7891723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:53.894637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f1373d-6c85-4730-8306-f5d7e7891723 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:53.903088 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54770173-1af8-4ffc-83d3-df7e8f2c1f72 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:53.904271 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 204/395] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:53.911350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1373d-6c85-4730-8306-f5d7e7891723 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:53.990993 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b9cd4-2755-4264-b9ec-f2c861b206ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:53.991867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b9cd4-2755-4264-b9ec-f2c861b206ab None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:53.993059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b9cd4-2755-4264-b9ec-f2c861b206ab None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:53.995599 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b9cd4-2755-4264-b9ec-f2c861b206ab None admin] PATH_INFO: `/v3/groups/4c15fded3d4f4a50a54b7eab156e431e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:53.997489 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b9cd4-2755-4264-b9ec-f2c861b206ab None admin] RBAC: Authorizing `identity:get_group(group_id=4c15fded3d4f4a50a54b7eab156e431e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:53.998947 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b9cd4-2755-4264-b9ec-f2c861b206ab None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:54.003930 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 192/396] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Jul 14 10:00:53 2022] GET /identity/v3/groups/4c15fded3d4f4a50a54b7eab156e431e => generated 226 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:54.018419 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b9cd4-2755-4264-b9ec-f2c861b206ab None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:54.053285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:54.067612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1373d-6c85-4730-8306-f5d7e7891723 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:54.119409 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844be61-dc14-40ac-a0ff-56320e4b5c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:54.120293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844be61-dc14-40ac-a0ff-56320e4b5c90 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:54.120555 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844be61-dc14-40ac-a0ff-56320e4b5c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:54.120830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844be61-dc14-40ac-a0ff-56320e4b5c90 None admin] PATH_INFO: `/v3/groups/4c15fded3d4f4a50a54b7eab156e431e` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:54.122052 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844be61-dc14-40ac-a0ff-56320e4b5c90 None admin] RBAC: Authorizing `identity:get_group(group_id=4c15fded3d4f4a50a54b7eab156e431e)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:54.123497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844be61-dc14-40ac-a0ff-56320e4b5c90 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:54.125575 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 205/397] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Jul 14 10:00:53 2022] GET /identity/v3/groups/4c15fded3d4f4a50a54b7eab156e431e => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:54.131004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844be61-dc14-40ac-a0ff-56320e4b5c90 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:54.227915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b9cd4-2755-4264-b9ec-f2c861b206ab None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:54.274978 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae95292-650d-4e3a-a46c-d1f1f3e0204b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:54.276213 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae95292-650d-4e3a-a46c-d1f1f3e0204b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:54.276498 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae95292-650d-4e3a-a46c-d1f1f3e0204b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:54.276776 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae95292-650d-4e3a-a46c-d1f1f3e0204b None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:54.278746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae95292-650d-4e3a-a46c-d1f1f3e0204b None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:54.280452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae95292-650d-4e3a-a46c-d1f1f3e0204b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:54.284172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 193/398] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:54 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:54.286132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a2989-1b03-4d9d-89f8-a1a9ec2134b3 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:54.286567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a2989-1b03-4d9d-89f8-a1a9ec2134b3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:54.287254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1a2989-1b03-4d9d-89f8-a1a9ec2134b3 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:54.293136 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 194/399] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:54 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:54.294364 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a2989-1b03-4d9d-89f8-a1a9ec2134b3 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:54.297410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844be61-dc14-40ac-a0ff-56320e4b5c90 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:54.305258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:54.338665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3810211-1fe1-458c-912e-36108ae979ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:54.339536 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3810211-1fe1-458c-912e-36108ae979ae None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:54.339817 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3810211-1fe1-458c-912e-36108ae979ae None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:54.340089 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3810211-1fe1-458c-912e-36108ae979ae None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:54.341311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3810211-1fe1-458c-912e-36108ae979ae None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:54.342745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3810211-1fe1-458c-912e-36108ae979ae None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:54.345031 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 206/400] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:54 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:54.350700 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea64df-2b19-4850-a91d-f31d355beb84 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:54.350984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea64df-2b19-4850-a91d-f31d355beb84 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:54.351266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ea64df-2b19-4850-a91d-f31d355beb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:54.368435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-87ea64df-2b19-4850-a91d-f31d355beb84 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:54.369910 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-87ea64df-2b19-4850-a91d-f31d355beb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:54.394022 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:54.416027 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1a2989-1b03-4d9d-89f8-a1a9ec2134b3 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:54.442140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 207/401] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:54.443651 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bd139-51d0-45e3-af44-c3bfe9f86eb8 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:54.444095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bd139-51d0-45e3-af44-c3bfe9f86eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:54.444524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9bd139-51d0-45e3-af44-c3bfe9f86eb8 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:54.446228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 208/402] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:54.447055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bd139-51d0-45e3-af44-c3bfe9f86eb8 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:54.462627 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd55c1-7acb-48fd-8777-183c7883d920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:54.463549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd55c1-7acb-48fd-8777-183c7883d920 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:54.463821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd55c1-7acb-48fd-8777-183c7883d920 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:54.464095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd55c1-7acb-48fd-8777-183c7883d920 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c/groups/4c15fded3d4f4a50a54b7eab156e431e/roles/f4e1d6f34f674f539ea7f2564646c0a5` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:54.466633 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd55c1-7acb-48fd-8777-183c7883d920 None admin] RBAC: Authorizing `identity:create_grant(project_id=ee948cfa1f0c4c65a69fc5204872db0c, group_id=4c15fded3d4f4a50a54b7eab156e431e, role_id=f4e1d6f34f674f539ea7f2564646c0a5)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:54.468199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd55c1-7acb-48fd-8777-183c7883d920 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:54.489038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 195/403] 10.222.0.61 () {58 vars in 1442 bytes} [Thu Jul 14 10:00:54 2022] PUT /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c/groups/4c15fded3d4f4a50a54b7eab156e431e/roles/f4e1d6f34f674f539ea7f2564646c0a5 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:00:54.489038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f3d4e-aded-4c39-8741-7c29f7066dd4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:54.489038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f3d4e-aded-4c39-8741-7c29f7066dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:54.489038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5f3d4e-aded-4c39-8741-7c29f7066dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:54.502873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-cf5f3d4e-aded-4c39-8741-7c29f7066dd4 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:54.504128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cf5f3d4e-aded-4c39-8741-7c29f7066dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:54.583912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:54.596963 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 196/404] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:54.597842 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f3d4e-aded-4c39-8741-7c29f7066dd4 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:54.599471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9bd139-51d0-45e3-af44-c3bfe9f86eb8 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:54.653810 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6553735-53b4-4861-92be-25f5c22113b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:54.655264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6553735-53b4-4861-92be-25f5c22113b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:54.655515 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6553735-53b4-4861-92be-25f5c22113b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:54.655707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6553735-53b4-4861-92be-25f5c22113b5 None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c/groups/4c15fded3d4f4a50a54b7eab156e431e/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:54.657753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6553735-53b4-4861-92be-25f5c22113b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=00a316c6194d45b392adff3ea7182a1c, group_id=4c15fded3d4f4a50a54b7eab156e431e, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:54.659211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6553735-53b4-4861-92be-25f5c22113b5 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:54.674730 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 209/405] 10.222.0.61 () {58 vars in 1442 bytes} [Thu Jul 14 10:00:54 2022] PUT /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c/groups/4c15fded3d4f4a50a54b7eab156e431e/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:00:54.675509 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6553735-53b4-4861-92be-25f5c22113b5 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:54.697354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:54.725830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5f3d4e-aded-4c39-8741-7c29f7066dd4 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:54.797292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4af56b-6531-40ad-8910-3743a235d733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:54.798420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4af56b-6531-40ad-8910-3743a235d733 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:54.798880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4af56b-6531-40ad-8910-3743a235d733 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:54.799345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4af56b-6531-40ad-8910-3743a235d733 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:54.823339 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4af56b-6531-40ad-8910-3743a235d733 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-dc4af56b-6531-40ad-8910-3743a235d733 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:54.840540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:54.841563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:54.842363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:54.842363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:54.842363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:54.842363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:54.842363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:54.842363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:54.842363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:54.842363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:54.842363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:54.842363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:54.842363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:54.842363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:54.847370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 197/406] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:00:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:54.855622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4af56b-6531-40ad-8910-3743a235d733 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:54.869306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6553735-53b4-4861-92be-25f5c22113b5 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:54.917564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107b5d0-f73c-4e64-b62f-bfb639e7e7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:54.919271 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107b5d0-f73c-4e64-b62f-bfb639e7e7eb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:54.919715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107b5d0-f73c-4e64-b62f-bfb639e7e7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:54.920145 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107b5d0-f73c-4e64-b62f-bfb639e7e7eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:54.926202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:54.936593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c107b5d0-f73c-4e64-b62f-bfb639e7e7eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-c107b5d0-f73c-4e64-b62f-bfb639e7e7eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:54.947168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:54.948282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:54.949341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:54.949341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:54.949341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:54.949341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:54.949341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:54.949341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:54.949341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:54.949341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:54.949341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:54.949341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:54.949341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:54.949341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:54.951303 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 210/407] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:00:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:54.957875 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107b5d0-f73c-4e64-b62f-bfb639e7e7eb None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:54.979432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4af56b-6531-40ad-8910-3743a235d733 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:55.026262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11522e5d-63b3-4d7c-b618-cf7b749ff689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:55.027177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11522e5d-63b3-4d7c-b618-cf7b749ff689 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:55.027458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11522e5d-63b3-4d7c-b618-cf7b749ff689 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:55.027733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11522e5d-63b3-4d7c-b618-cf7b749ff689 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:55.028541 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11522e5d-63b3-4d7c-b618-cf7b749ff689 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:55.028849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11522e5d-63b3-4d7c-b618-cf7b749ff689 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:55.031568 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11522e5d-63b3-4d7c-b618-cf7b749ff689 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:55.041735 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 198/408] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:55.055800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:55.060147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11522e5d-63b3-4d7c-b618-cf7b749ff689 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:55.082399 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107b5d0-f73c-4e64-b62f-bfb639e7e7eb None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:55.124446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bbea4-e51f-4139-9b41-7bf6c21ad8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:55.127778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bbea4-e51f-4139-9b41-7bf6c21ad8fe None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:55.128059 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bbea4-e51f-4139-9b41-7bf6c21ad8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:55.128333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bbea4-e51f-4139-9b41-7bf6c21ad8fe None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:55.129119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169bbea4-e51f-4139-9b41-7bf6c21ad8fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:55.129433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169bbea4-e51f-4139-9b41-7bf6c21ad8fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:55.136952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169bbea4-e51f-4139-9b41-7bf6c21ad8fe None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:55.149936 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 211/409] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:00:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:55.155935 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bbea4-e51f-4139-9b41-7bf6c21ad8fe None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:55.232576 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11522e5d-63b3-4d7c-b618-cf7b749ff689 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:55.277307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:55.315384 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0afb2-3fa3-4824-a28c-c2d0a84037d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:55.316418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0afb2-3fa3-4824-a28c-c2d0a84037d5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:55.316693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0afb2-3fa3-4824-a28c-c2d0a84037d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:55.317009 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c0afb2-3fa3-4824-a28c-c2d0a84037d5 None admin] PATH_INFO: `/v3/groups/75718cd66c97465daa0dfd23189109d5` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:55.319707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0afb2-3fa3-4824-a28c-c2d0a84037d5 None admin] RBAC: Authorizing `identity:get_group(group_id=75718cd66c97465daa0dfd23189109d5)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:55.321179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c0afb2-3fa3-4824-a28c-c2d0a84037d5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:55.327268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 199/410] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Jul 14 10:00:55 2022] GET /identity/v3/groups/75718cd66c97465daa0dfd23189109d5 => generated 229 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:55.328505 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0afb2-3fa3-4824-a28c-c2d0a84037d5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:55.357027 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bbea4-e51f-4139-9b41-7bf6c21ad8fe None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:55.393333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ec45e-e925-43b7-961a-0dc2ac92a427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:55.394232 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ec45e-e925-43b7-961a-0dc2ac92a427 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:55.394757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ec45e-e925-43b7-961a-0dc2ac92a427 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:55.394920 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ec45e-e925-43b7-961a-0dc2ac92a427 None admin] PATH_INFO: `/v3/groups/4c15fded3d4f4a50a54b7eab156e431e` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:55.396136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ec45e-e925-43b7-961a-0dc2ac92a427 None admin] RBAC: Authorizing `identity:get_group(group_id=4c15fded3d4f4a50a54b7eab156e431e)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:55.397623 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ec45e-e925-43b7-961a-0dc2ac92a427 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:55.399766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 212/411] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Jul 14 10:00:55 2022] GET /identity/v3/groups/4c15fded3d4f4a50a54b7eab156e431e => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:55.409959 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ec45e-e925-43b7-961a-0dc2ac92a427 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:55.427268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:55.431123 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c0afb2-3fa3-4824-a28c-c2d0a84037d5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:55.465165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c0246-3f4f-454a-85a1-9b0c99f83f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:55.466020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c0246-3f4f-454a-85a1-9b0c99f83f34 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:55.466297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c0246-3f4f-454a-85a1-9b0c99f83f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:55.466565 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c0246-3f4f-454a-85a1-9b0c99f83f34 None admin] PATH_INFO: `/v3/projects/4677fbdc96dd49fca8554ea63b0d5a7c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:55.467743 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c0246-3f4f-454a-85a1-9b0c99f83f34 None admin] RBAC: Authorizing `identity:get_project(project_id=4677fbdc96dd49fca8554ea63b0d5a7c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:55.469768 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c0246-3f4f-454a-85a1-9b0c99f83f34 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:55.476095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 200/412] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:55 2022] GET /identity/v3/projects/4677fbdc96dd49fca8554ea63b0d5a7c => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:55.477265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c0246-3f4f-454a-85a1-9b0c99f83f34 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:55.554584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ec45e-e925-43b7-961a-0dc2ac92a427 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:55.597051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f86f4-13c9-41da-af65-95883e10ab8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:55.597051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f86f4-13c9-41da-af65-95883e10ab8a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:55.597051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f86f4-13c9-41da-af65-95883e10ab8a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:55.597051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f86f4-13c9-41da-af65-95883e10ab8a None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:55.597425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f86f4-13c9-41da-af65-95883e10ab8a None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:55.597425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f86f4-13c9-41da-af65-95883e10ab8a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:55.599086 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 213/413] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:55 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:55.610392 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f86f4-13c9-41da-af65-95883e10ab8a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:55.619146 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:55.665909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c0246-3f4f-454a-85a1-9b0c99f83f34 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:55.701476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab38e2-1d20-4725-a453-3f00098f8418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:55.703259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab38e2-1d20-4725-a453-3f00098f8418 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:55.703753 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab38e2-1d20-4725-a453-3f00098f8418 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:55.704222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab38e2-1d20-4725-a453-3f00098f8418 None admin] PATH_INFO: `/v3/projects/4677fbdc96dd49fca8554ea63b0d5a7c/groups/75718cd66c97465daa0dfd23189109d5/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:55.706470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab38e2-1d20-4725-a453-3f00098f8418 None admin] RBAC: Authorizing `identity:create_grant(project_id=4677fbdc96dd49fca8554ea63b0d5a7c, group_id=75718cd66c97465daa0dfd23189109d5, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:55.708113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab38e2-1d20-4725-a453-3f00098f8418 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:55.742414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 201/414] 10.222.0.61 () {58 vars in 1442 bytes} [Thu Jul 14 10:00:55 2022] PUT /identity/v3/projects/4677fbdc96dd49fca8554ea63b0d5a7c/groups/75718cd66c97465daa0dfd23189109d5/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:00:55.768067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:55.797406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f86f4-13c9-41da-af65-95883e10ab8a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:55.841884 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c68ff-a738-4832-b46b-15e93f7d7f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:55.842776 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c68ff-a738-4832-b46b-15e93f7d7f37 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:55.843051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c68ff-a738-4832-b46b-15e93f7d7f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:55.843352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c68ff-a738-4832-b46b-15e93f7d7f37 None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c/groups/4c15fded3d4f4a50a54b7eab156e431e/roles/f4e1d6f34f674f539ea7f2564646c0a5` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:55.847200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c68ff-a738-4832-b46b-15e93f7d7f37 None admin] RBAC: Authorizing `identity:create_grant(project_id=00a316c6194d45b392adff3ea7182a1c, group_id=4c15fded3d4f4a50a54b7eab156e431e, role_id=f4e1d6f34f674f539ea7f2564646c0a5)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:55.848598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187c68ff-a738-4832-b46b-15e93f7d7f37 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:55.882823 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 214/415] 10.222.0.61 () {58 vars in 1442 bytes} [Thu Jul 14 10:00:55 2022] PUT /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c/groups/4c15fded3d4f4a50a54b7eab156e431e/roles/f4e1d6f34f674f539ea7f2564646c0a5 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:00:55.992474 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9c10d-9a2b-407a-89f0-fb6aadd19c11 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:55.992832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9c10d-9a2b-407a-89f0-fb6aadd19c11 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:55.993069 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f9c10d-9a2b-407a-89f0-fb6aadd19c11 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:55.994488 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 202/416] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:56.000838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35475e9f-73e3-420d-a8d7-e6fb0c89f4cf None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:56.001178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35475e9f-73e3-420d-a8d7-e6fb0c89f4cf None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:56.001462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35475e9f-73e3-420d-a8d7-e6fb0c89f4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:56.019667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-35475e9f-73e3-420d-a8d7-e6fb0c89f4cf None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:56.020894 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-35475e9f-73e3-420d-a8d7-e6fb0c89f4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:56.099144 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 215/417] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:56.122400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9c10d-9a2b-407a-89f0-fb6aadd19c11 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:56.245745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f9c10d-9a2b-407a-89f0-fb6aadd19c11 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:56.267602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:56.296901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a3053-dc22-4c25-9b9f-dbc34e486dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:56.297737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a3053-dc22-4c25-9b9f-dbc34e486dde None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:56.298020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a3053-dc22-4c25-9b9f-dbc34e486dde None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:56.298293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2a3053-dc22-4c25-9b9f-dbc34e486dde None admin] PATH_INFO: `/v3/roles/member` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:56.313104 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2a3053-dc22-4c25-9b9f-dbc34e486dde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-6b2a3053-dc22-4c25-9b9f-dbc34e486dde None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:56.321172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:56.322053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:56.322846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:56.322846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:56.322846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:56.322846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:56.322846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:56.322846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:56.322846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:56.322846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:56.322846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:56.322846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:56.322846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:56.322846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:56.326580 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 203/418] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:56.332368 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35475e9f-73e3-420d-a8d7-e6fb0c89f4cf None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:56.433527 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:56.483097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35475e9f-73e3-420d-a8d7-e6fb0c89f4cf None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:56.527955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3d59e-a567-41e9-89bd-39fbcbd2daa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:56.528994 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3d59e-a567-41e9-89bd-39fbcbd2daa9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:56.529501 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3d59e-a567-41e9-89bd-39fbcbd2daa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:56.529965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3d59e-a567-41e9-89bd-39fbcbd2daa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:56.531488 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3d59e-a567-41e9-89bd-39fbcbd2daa9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:56.532022 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3d59e-a567-41e9-89bd-39fbcbd2daa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:56.533661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c3d59e-a567-41e9-89bd-39fbcbd2daa9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:56.542993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 216/419] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:56.548288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a3053-dc22-4c25-9b9f-dbc34e486dde None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:56.698010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2a3053-dc22-4c25-9b9f-dbc34e486dde None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:56.741010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba99860-aa99-4f4c-ae3a-290d69d9f650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:56.742024 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba99860-aa99-4f4c-ae3a-290d69d9f650 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:56.742528 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba99860-aa99-4f4c-ae3a-290d69d9f650 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:56.743014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba99860-aa99-4f4c-ae3a-290d69d9f650 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:56.798259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:56.832410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba99860-aa99-4f4c-ae3a-290d69d9f650 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:56.833873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba99860-aa99-4f4c-ae3a-290d69d9f650 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:56.897920 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5eda-fc40-4d6f-9789-04c2c16ee6ea None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:56.898258 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5eda-fc40-4d6f-9789-04c2c16ee6ea None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:56.898479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd5eda-fc40-4d6f-9789-04c2c16ee6ea None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:56.900086 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 217/420] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:00:56.909565 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05166360-60cb-472e-9350-4ea2b0e073b5 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:56.909849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05166360-60cb-472e-9350-4ea2b0e073b5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:56.910126 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05166360-60cb-472e-9350-4ea2b0e073b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:56.936699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-05166360-60cb-472e-9350-4ea2b0e073b5 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:56.937926 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-05166360-60cb-472e-9350-4ea2b0e073b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:57.012144 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 204/421] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:57.017567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba99860-aa99-4f4c-ae3a-290d69d9f650 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:57.045530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 218/422] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:57.068221 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05166360-60cb-472e-9350-4ea2b0e073b5 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:57.147708 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:57.147908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:57.178151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba99860-aa99-4f4c-ae3a-290d69d9f650 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:57.194171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05166360-60cb-472e-9350-4ea2b0e073b5 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:57.217165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac7fcc-89ef-4981-b6c8-5089dcddca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:57.219042 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac7fcc-89ef-4981-b6c8-5089dcddca2f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:57.219393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac7fcc-89ef-4981-b6c8-5089dcddca2f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:57.219672 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac7fcc-89ef-4981-b6c8-5089dcddca2f None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:57.221055 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac7fcc-89ef-4981-b6c8-5089dcddca2f None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:57.222610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac7fcc-89ef-4981-b6c8-5089dcddca2f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:57.224679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 205/423] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:57 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:57.225893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b106e4-0df2-4162-a73c-ae387df72451 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:57.226179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b106e4-0df2-4162-a73c-ae387df72451 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:57.226454 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b106e4-0df2-4162-a73c-ae387df72451 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:57.228143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 206/424] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:57.229572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b106e4-0df2-4162-a73c-ae387df72451 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:57.242275 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dffb1a-3cfd-4b40-acdc-2a9adecedd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:57.243195 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dffb1a-3cfd-4b40-acdc-2a9adecedd9d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:57.244007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dffb1a-3cfd-4b40-acdc-2a9adecedd9d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:57.250522 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dffb1a-3cfd-4b40-acdc-2a9adecedd9d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:57.266025 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dffb1a-3cfd-4b40-acdc-2a9adecedd9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-32dffb1a-3cfd-4b40-acdc-2a9adecedd9d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:57.274138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:57.275035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:57.276228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:57.276228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:57.276228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:57.276228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:57.276228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:57.276228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:57.276228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:57.276228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:57.276228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:57.276228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:57.276228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:57.276228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:57.276228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 219/425] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:00:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:00:57.277181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620590a6-8d04-40b4-a459-82a59f351a30 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:57.283279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620590a6-8d04-40b4-a459-82a59f351a30 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:57.283584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620590a6-8d04-40b4-a459-82a59f351a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:57.300902 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-620590a6-8d04-40b4-a459-82a59f351a30 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:57.302144 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-620590a6-8d04-40b4-a459-82a59f351a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:57.352053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b106e4-0df2-4162-a73c-ae387df72451 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:57.370784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 220/426] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:57.371727 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620590a6-8d04-40b4-a459-82a59f351a30 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:57.395150 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfc18e-03e2-4d48-99f8-4db306a648ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:57.396466 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfc18e-03e2-4d48-99f8-4db306a648ce None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:57.402117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfc18e-03e2-4d48-99f8-4db306a648ce None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:57.402117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccfc18e-03e2-4d48-99f8-4db306a648ce None admin] PATH_INFO: `/v3/groups/4c15fded3d4f4a50a54b7eab156e431e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:57.403374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccfc18e-03e2-4d48-99f8-4db306a648ce None admin] RBAC: Authorizing `identity:get_group(group_id=4c15fded3d4f4a50a54b7eab156e431e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:57.404169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccfc18e-03e2-4d48-99f8-4db306a648ce None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:57.405892 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 207/427] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Jul 14 10:00:57 2022] GET /identity/v3/groups/4c15fded3d4f4a50a54b7eab156e431e => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:57.409668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfc18e-03e2-4d48-99f8-4db306a648ce None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:57.429326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:57.432046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:57.486604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620590a6-8d04-40b4-a459-82a59f351a30 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:57.522707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85071d4f-be19-4b55-8827-cf8e011a4506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:57.523298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85071d4f-be19-4b55-8827-cf8e011a4506 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:57.523579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85071d4f-be19-4b55-8827-cf8e011a4506 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:57.523855 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85071d4f-be19-4b55-8827-cf8e011a4506 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:57.524648 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85071d4f-be19-4b55-8827-cf8e011a4506 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:57.524913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85071d4f-be19-4b55-8827-cf8e011a4506 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:57.526320 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85071d4f-be19-4b55-8827-cf8e011a4506 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:57.536533 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccfc18e-03e2-4d48-99f8-4db306a648ce None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:57.539438 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 221/428] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:00:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:57.540202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85071d4f-be19-4b55-8827-cf8e011a4506 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:57.591022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1400afa-9775-4504-928c-e244d303125d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:57.591820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1400afa-9775-4504-928c-e244d303125d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:57.592097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1400afa-9775-4504-928c-e244d303125d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:57.592370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1400afa-9775-4504-928c-e244d303125d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:57.613882 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1400afa-9775-4504-928c-e244d303125d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-c1400afa-9775-4504-928c-e244d303125d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:57.625351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:57.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:57.627175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:57.627175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:57.627175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:57.627175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:57.627175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:57.627175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:57.627175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:57.627175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:57.627175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:57.627175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:57.627175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 14 10:00:57.627175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:57.633334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 208/429] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:00:57.633334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1400afa-9775-4504-928c-e244d303125d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:57.671730 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85071d4f-be19-4b55-8827-cf8e011a4506 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:57.685967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:57.705959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:57.723225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247a28e-729e-4327-a228-e102e1224f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:57.724529 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247a28e-729e-4327-a228-e102e1224f2c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:57.725041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247a28e-729e-4327-a228-e102e1224f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:57.725519 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247a28e-729e-4327-a228-e102e1224f2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:57.727089 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247a28e-729e-4327-a228-e102e1224f2c None admin] RBAC: Adding query filter params (group.id=4c15fded3d4f4a50a54b7eab156e431e, role.id=3d9688bbbc15422cbfc59ded082fbe41, scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:57.727641 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247a28e-729e-4327-a228-e102e1224f2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:57.729803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247a28e-729e-4327-a228-e102e1224f2c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:57.739736 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 222/430] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Jul 14 10:00:57 2022] GET /identity/v3/role_assignments?group.id=4c15fded3d4f4a50a54b7eab156e431e&scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c&role.id=3d9688bbbc15422cbfc59ded082fbe41 => generated 611 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:57.740415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1400afa-9775-4504-928c-e244d303125d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:57.741072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247a28e-729e-4327-a228-e102e1224f2c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:57.788958 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473abad-07d7-4114-ab2d-c2b4fea8ad42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:57.790044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473abad-07d7-4114-ab2d-c2b4fea8ad42 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:57.790986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473abad-07d7-4114-ab2d-c2b4fea8ad42 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:57.791447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473abad-07d7-4114-ab2d-c2b4fea8ad42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:57.857007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247a28e-729e-4327-a228-e102e1224f2c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:57.877897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473abad-07d7-4114-ab2d-c2b4fea8ad42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:57.879647 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473abad-07d7-4114-ab2d-c2b4fea8ad42 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:57.894775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf6218-d5e3-4a2e-9e50-941de9bc4303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:57.895557 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf6218-d5e3-4a2e-9e50-941de9bc4303 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:57.895913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf6218-d5e3-4a2e-9e50-941de9bc4303 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:57.896285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf6218-d5e3-4a2e-9e50-941de9bc4303 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:57.897161 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf6218-d5e3-4a2e-9e50-941de9bc4303 None admin] RBAC: Adding query filter params (name=member) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:57.897540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf6218-d5e3-4a2e-9e50-941de9bc4303 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:57.899247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf6218-d5e3-4a2e-9e50-941de9bc4303 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:57.913179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 223/431] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:00:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:57.915703 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf6218-d5e3-4a2e-9e50-941de9bc4303 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:57.954775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:57.963576 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:58.002053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 209/432] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:58.008059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473abad-07d7-4114-ab2d-c2b4fea8ad42 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:58.025262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf6218-d5e3-4a2e-9e50-941de9bc4303 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:58.063420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a028ac-6883-4de5-b048-b9de7f669c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:58.064132 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a028ac-6883-4de5-b048-b9de7f669c8e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:58.064326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a028ac-6883-4de5-b048-b9de7f669c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:58.064600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a028ac-6883-4de5-b048-b9de7f669c8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:58.133682 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473abad-07d7-4114-ab2d-c2b4fea8ad42 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:58.137384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a028ac-6883-4de5-b048-b9de7f669c8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:58.138855 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a028ac-6883-4de5-b048-b9de7f669c8e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:58.172610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdaae04-983c-4a67-8ee4-dcef8d5482c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:58.173901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaae04-983c-4a67-8ee4-dcef8d5482c2 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:58.174401 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaae04-983c-4a67-8ee4-dcef8d5482c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:58.174874 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdaae04-983c-4a67-8ee4-dcef8d5482c2 None admin] PATH_INFO: `/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:58.176307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdaae04-983c-4a67-8ee4-dcef8d5482c2 None admin] RBAC: Authorizing `identity:get_project(project_id=ee948cfa1f0c4c65a69fc5204872db0c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:58.177988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdaae04-983c-4a67-8ee4-dcef8d5482c2 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:58.184422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 210/433] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:58 2022] GET /identity/v3/projects/ee948cfa1f0c4c65a69fc5204872db0c => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:58.186102 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdaae04-983c-4a67-8ee4-dcef8d5482c2 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:58.198477 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:58.230873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:58.251247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 224/434] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:00:58.254116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a028ac-6883-4de5-b048-b9de7f669c8e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:58.331079 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdaae04-983c-4a67-8ee4-dcef8d5482c2 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:58.379641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3236f-fdb2-49be-8daf-a08eeeb06d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:58.380106 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3236f-fdb2-49be-8daf-a08eeeb06d84 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:58.381392 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3236f-fdb2-49be-8daf-a08eeeb06d84 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:58.381670 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3236f-fdb2-49be-8daf-a08eeeb06d84 None admin] PATH_INFO: `/v3/groups/4c15fded3d4f4a50a54b7eab156e431e` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:58.383179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3236f-fdb2-49be-8daf-a08eeeb06d84 None admin] RBAC: Authorizing `identity:get_group(group_id=4c15fded3d4f4a50a54b7eab156e431e)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:58.385302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3236f-fdb2-49be-8daf-a08eeeb06d84 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:58.389778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 211/435] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Jul 14 10:00:58 2022] GET /identity/v3/groups/4c15fded3d4f4a50a54b7eab156e431e => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:58.392099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a028ac-6883-4de5-b048-b9de7f669c8e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:58.394755 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3236f-fdb2-49be-8daf-a08eeeb06d84 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:58.436774 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128bdd4-2435-441b-adf0-ae120468124a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:58.437675 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128bdd4-2435-441b-adf0-ae120468124a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:58.439140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128bdd4-2435-441b-adf0-ae120468124a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:58.439140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128bdd4-2435-441b-adf0-ae120468124a None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:58.439410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c128bdd4-2435-441b-adf0-ae120468124a None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:58.441143 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c128bdd4-2435-441b-adf0-ae120468124a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:58.443158 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 225/436] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:58 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:58.444397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ff720-3e0b-4c8f-a208-90c823965f38 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:58.444678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ff720-3e0b-4c8f-a208-90c823965f38 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:58.445961 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301ff720-3e0b-4c8f-a208-90c823965f38 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:58.447453 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 226/437] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:58.450740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ff720-3e0b-4c8f-a208-90c823965f38 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:58.456050 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:58.528510 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3236f-fdb2-49be-8daf-a08eeeb06d84 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:58.540566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:58.574358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335e6dc-9f4a-4a04-9dfc-e8d3237085e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:58.575022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335e6dc-9f4a-4a04-9dfc-e8d3237085e9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:58.575302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335e6dc-9f4a-4a04-9dfc-e8d3237085e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:58.575575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335e6dc-9f4a-4a04-9dfc-e8d3237085e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:58.576409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335e6dc-9f4a-4a04-9dfc-e8d3237085e9 None admin] RBAC: Adding query filter params (group.id=4c15fded3d4f4a50a54b7eab156e431e, role.id=f4e1d6f34f674f539ea7f2564646c0a5, scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:58.577622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335e6dc-9f4a-4a04-9dfc-e8d3237085e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:58.578852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335e6dc-9f4a-4a04-9dfc-e8d3237085e9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:58.584883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 212/438] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Jul 14 10:00:58 2022] GET /identity/v3/role_assignments?group.id=4c15fded3d4f4a50a54b7eab156e431e&scope.project.id=ee948cfa1f0c4c65a69fc5204872db0c&role.id=f4e1d6f34f674f539ea7f2564646c0a5 => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:58.593167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5d86f-b1a9-4b64-80dd-4a24ad1cfa1c None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:58.593167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5d86f-b1a9-4b64-80dd-4a24ad1cfa1c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:58.593167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5d86f-b1a9-4b64-80dd-4a24ad1cfa1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:58.593868 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301ff720-3e0b-4c8f-a208-90c823965f38 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:58.606184 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d0c5d86f-b1a9-4b64-80dd-4a24ad1cfa1c None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:58.607419 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d0c5d86f-b1a9-4b64-80dd-4a24ad1cfa1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:58.638373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7fdc27-1e95-468c-82d9-e59ded1b5150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:58.639862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fdc27-1e95-468c-82d9-e59ded1b5150 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:58.640131 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fdc27-1e95-468c-82d9-e59ded1b5150 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:58.640406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7fdc27-1e95-468c-82d9-e59ded1b5150 None admin] PATH_INFO: `/v3/groups/4c15fded3d4f4a50a54b7eab156e431e` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:58.641737 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fdc27-1e95-468c-82d9-e59ded1b5150 None admin] RBAC: Authorizing `identity:get_group(group_id=4c15fded3d4f4a50a54b7eab156e431e)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:58.643217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7fdc27-1e95-468c-82d9-e59ded1b5150 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:58.645303 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 227/439] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Jul 14 10:00:58 2022] GET /identity/v3/groups/4c15fded3d4f4a50a54b7eab156e431e => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:58.646597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4c6f6-7b53-4ff4-86c5-aff9d3b4b331 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:58.646883 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4c6f6-7b53-4ff4-86c5-aff9d3b4b331 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:58.647159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4c6f6-7b53-4ff4-86c5-aff9d3b4b331 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:58.648405 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 228/440] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:00:58.653787 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2c424-0b2a-4c8d-a130-0425378cb84e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:58.654066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2c424-0b2a-4c8d-a130-0425378cb84e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:58.654348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b2c424-0b2a-4c8d-a130-0425378cb84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:58.672432 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f1b2c424-0b2a-4c8d-a130-0425378cb84e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:00:58.673698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f1b2c424-0b2a-4c8d-a130-0425378cb84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:00:58.692297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 213/441] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:00:58.693209 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5d86f-b1a9-4b64-80dd-4a24ad1cfa1c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:58.761005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 229/442] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:00:58.761587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2c424-0b2a-4c8d-a130-0425378cb84e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:58.763649 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:58.820536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:58.823487 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5d86f-b1a9-4b64-80dd-4a24ad1cfa1c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:58.864141 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a4c01-9111-4cfb-a577-f8d9fda55c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:58.864329 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a4c01-9111-4cfb-a577-f8d9fda55c19 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:58.864560 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a4c01-9111-4cfb-a577-f8d9fda55c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:58.864833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08a4c01-9111-4cfb-a577-f8d9fda55c19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:58.865784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a4c01-9111-4cfb-a577-f8d9fda55c19 None admin] RBAC: Adding query filter params (group.id=4c15fded3d4f4a50a54b7eab156e431e, role.id=3d9688bbbc15422cbfc59ded082fbe41, scope.project.id=00a316c6194d45b392adff3ea7182a1c) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:58.866101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a4c01-9111-4cfb-a577-f8d9fda55c19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:58.867888 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08a4c01-9111-4cfb-a577-f8d9fda55c19 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:58.874348 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 214/443] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Jul 14 10:00:58 2022] GET /identity/v3/role_assignments?group.id=4c15fded3d4f4a50a54b7eab156e431e&scope.project.id=00a316c6194d45b392adff3ea7182a1c&role.id=3d9688bbbc15422cbfc59ded082fbe41 => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:00:58.877780 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a4c01-9111-4cfb-a577-f8d9fda55c19 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:58.898019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b2c424-0b2a-4c8d-a130-0425378cb84e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:58.945741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebe971-b9aa-4523-89c7-d474b06a7e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:58.946563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebe971-b9aa-4523-89c7-d474b06a7e3f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:58.946837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebe971-b9aa-4523-89c7-d474b06a7e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:58.947114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eebe971-b9aa-4523-89c7-d474b06a7e3f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:58.966164 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eebe971-b9aa-4523-89c7-d474b06a7e3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-2eebe971-b9aa-4523-89c7-d474b06a7e3f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:58.976772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:58.977849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:58.978762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:58.978762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:58.978762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:58.978762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:58.978762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:58.978762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:58.978762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:58.978762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:58.978762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:58.978762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:58.978762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:00:58.978762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:00:58.980979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 230/444] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:00:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:58.989211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebe971-b9aa-4523-89c7-d474b06a7e3f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:59.024578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08a4c01-9111-4cfb-a577-f8d9fda55c19 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:59.066355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68816b05-c63e-46e3-930d-98e0fdfbf899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:59.066559 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:59.067626 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816b05-c63e-46e3-930d-98e0fdfbf899 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:59.068100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816b05-c63e-46e3-930d-98e0fdfbf899 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:59.068548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68816b05-c63e-46e3-930d-98e0fdfbf899 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:59.080069 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68816b05-c63e-46e3-930d-98e0fdfbf899 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-68816b05-c63e-46e3-930d-98e0fdfbf899 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:00:59.087867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:00:59.088991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:00:59.089844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:00:59.089844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:00:59.089844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:00:59.089844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:00:59.089844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:00:59.089844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:00:59.089844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:00:59.089844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:00:59.089844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:00:59.089844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:00:59.089844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:00:59.089844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:00:59.089844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 215/445] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:00:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:00:59.095762 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68816b05-c63e-46e3-930d-98e0fdfbf899 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:59.119973 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eebe971-b9aa-4523-89c7-d474b06a7e3f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:59.128231 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:59.158812 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459e37a-dfca-4020-8148-ec4360612dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:59.159492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459e37a-dfca-4020-8148-ec4360612dbb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:59.159775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459e37a-dfca-4020-8148-ec4360612dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:59.160055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459e37a-dfca-4020-8148-ec4360612dbb None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:59.160818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459e37a-dfca-4020-8148-ec4360612dbb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:59.162141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459e37a-dfca-4020-8148-ec4360612dbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:59.163538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459e37a-dfca-4020-8148-ec4360612dbb None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:59.173463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 231/446] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:00:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:59.181141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459e37a-dfca-4020-8148-ec4360612dbb None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:59.222987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68816b05-c63e-46e3-930d-98e0fdfbf899 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:59.262274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e7cad-1740-4da8-9110-89502ae58b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:59.262945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e7cad-1740-4da8-9110-89502ae58b52 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:59.263233 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e7cad-1740-4da8-9110-89502ae58b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:59.263509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e7cad-1740-4da8-9110-89502ae58b52 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:59.264287 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e7cad-1740-4da8-9110-89502ae58b52 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:59.264594 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e7cad-1740-4da8-9110-89502ae58b52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:59.266016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e7cad-1740-4da8-9110-89502ae58b52 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:59.274097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 216/447] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:00:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:59.280028 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e7cad-1740-4da8-9110-89502ae58b52 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:59.293172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459e37a-dfca-4020-8148-ec4360612dbb None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:59.316911 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:59.328013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cffeb8-1c68-4a5e-863c-b71e03d7d4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:59.328666 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cffeb8-1c68-4a5e-863c-b71e03d7d4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:59.328974 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cffeb8-1c68-4a5e-863c-b71e03d7d4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:59.329256 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cffeb8-1c68-4a5e-863c-b71e03d7d4b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:59.366883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:59.382190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e7cad-1740-4da8-9110-89502ae58b52 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:59.394725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cffeb8-1c68-4a5e-863c-b71e03d7d4b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:59.396171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cffeb8-1c68-4a5e-863c-b71e03d7d4b3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:59.416000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadffe83-31e3-4eec-8bf3-984fa4803e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:59.416676 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadffe83-31e3-4eec-8bf3-984fa4803e55 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:59.416992 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadffe83-31e3-4eec-8bf3-984fa4803e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:59.417275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadffe83-31e3-4eec-8bf3-984fa4803e55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:59.484075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadffe83-31e3-4eec-8bf3-984fa4803e55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:59.485778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadffe83-31e3-4eec-8bf3-984fa4803e55 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:59.493029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 232/448] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:00:59.498462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cffeb8-1c68-4a5e-863c-b71e03d7d4b3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:59.521868 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:59.578652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:59.595296 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cffeb8-1c68-4a5e-863c-b71e03d7d4b3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:59.604144 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 217/449] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:00:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 14 10:00:59.609673 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadffe83-31e3-4eec-8bf3-984fa4803e55 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:59.628324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399e8b5-ac55-43cf-adfc-ed7290a0f3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:59.629204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399e8b5-ac55-43cf-adfc-ed7290a0f3f9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:59.629488 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399e8b5-ac55-43cf-adfc-ed7290a0f3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:59.629760 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d399e8b5-ac55-43cf-adfc-ed7290a0f3f9 None admin] PATH_INFO: `/v3/projects/00a316c6194d45b392adff3ea7182a1c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:59.630938 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399e8b5-ac55-43cf-adfc-ed7290a0f3f9 None admin] RBAC: Authorizing `identity:get_project(project_id=00a316c6194d45b392adff3ea7182a1c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:59.632412 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d399e8b5-ac55-43cf-adfc-ed7290a0f3f9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:59.634055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 233/450] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:59 2022] GET /identity/v3/projects/00a316c6194d45b392adff3ea7182a1c => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:59.639920 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399e8b5-ac55-43cf-adfc-ed7290a0f3f9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:59.706644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadffe83-31e3-4eec-8bf3-984fa4803e55 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:59.728293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:59.737918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d399e8b5-ac55-43cf-adfc-ed7290a0f3f9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:59.740463 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4977436-3f6f-4f38-b282-06362b7505a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:59.741345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4977436-3f6f-4f38-b282-06362b7505a0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:59.741620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4977436-3f6f-4f38-b282-06362b7505a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:59.741890 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4977436-3f6f-4f38-b282-06362b7505a0 None admin] PATH_INFO: `/v3/projects/4677fbdc96dd49fca8554ea63b0d5a7c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:59.743067 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4977436-3f6f-4f38-b282-06362b7505a0 None admin] RBAC: Authorizing `identity:get_project(project_id=4677fbdc96dd49fca8554ea63b0d5a7c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:59.744485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4977436-3f6f-4f38-b282-06362b7505a0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:59.746194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 218/451] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:00:59 2022] GET /identity/v3/projects/4677fbdc96dd49fca8554ea63b0d5a7c => generated 344 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:59.752083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4977436-3f6f-4f38-b282-06362b7505a0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:59.771594 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954b395-87a7-4c57-8ee6-dd8e3018706e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:59.772441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954b395-87a7-4c57-8ee6-dd8e3018706e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:59.772719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954b395-87a7-4c57-8ee6-dd8e3018706e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:59.773017 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5954b395-87a7-4c57-8ee6-dd8e3018706e None admin] PATH_INFO: `/v3/groups/4c15fded3d4f4a50a54b7eab156e431e` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:59.774174 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954b395-87a7-4c57-8ee6-dd8e3018706e None admin] RBAC: Authorizing `identity:get_group(group_id=4c15fded3d4f4a50a54b7eab156e431e)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:59.775598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5954b395-87a7-4c57-8ee6-dd8e3018706e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:59.777239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 234/452] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Jul 14 10:00:59 2022] GET /identity/v3/groups/4c15fded3d4f4a50a54b7eab156e431e => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:59.782834 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954b395-87a7-4c57-8ee6-dd8e3018706e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:59.806683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:00:59.850994 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4977436-3f6f-4f38-b282-06362b7505a0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:59.881327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5954b395-87a7-4c57-8ee6-dd8e3018706e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:00:59.886755 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4fd5a-61e2-41ee-a584-c7808312cd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:59.887550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4fd5a-61e2-41ee-a584-c7808312cd7a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:59.887828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4fd5a-61e2-41ee-a584-c7808312cd7a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:59.888101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4fd5a-61e2-41ee-a584-c7808312cd7a None admin] PATH_INFO: `/v3/groups/75718cd66c97465daa0dfd23189109d5` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:59.889864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f4fd5a-61e2-41ee-a584-c7808312cd7a None admin] RBAC: Authorizing `identity:get_group(group_id=75718cd66c97465daa0dfd23189109d5)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:59.891318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f4fd5a-61e2-41ee-a584-c7808312cd7a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:59.892957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 219/453] 10.222.0.61 () {56 vars in 1180 bytes} [Thu Jul 14 10:00:59 2022] GET /identity/v3/groups/75718cd66c97465daa0dfd23189109d5 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:00:59.898551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4fd5a-61e2-41ee-a584-c7808312cd7a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:00:59.915716 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf35acd-2a89-4725-827e-1d12e00f53f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:00:59.916374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf35acd-2a89-4725-827e-1d12e00f53f7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:00:59.916656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf35acd-2a89-4725-827e-1d12e00f53f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:00:59.916954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf35acd-2a89-4725-827e-1d12e00f53f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:00:59.917791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf35acd-2a89-4725-827e-1d12e00f53f7 None admin] RBAC: Adding query filter params (group.id=4c15fded3d4f4a50a54b7eab156e431e, role.id=f4e1d6f34f674f539ea7f2564646c0a5, scope.project.id=00a316c6194d45b392adff3ea7182a1c) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:00:59.918105 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf35acd-2a89-4725-827e-1d12e00f53f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:00:59.919539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf35acd-2a89-4725-827e-1d12e00f53f7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:00:59.925777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 235/454] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Jul 14 10:00:59 2022] GET /identity/v3/role_assignments?group.id=4c15fded3d4f4a50a54b7eab156e431e&scope.project.id=00a316c6194d45b392adff3ea7182a1c&role.id=f4e1d6f34f674f539ea7f2564646c0a5 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:00.001887 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4fd5a-61e2-41ee-a584-c7808312cd7a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:00.025705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:00.038229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff78c199-e611-46db-9074-9e930b0588a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:00.038899 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78c199-e611-46db-9074-9e930b0588a1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:00.039179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78c199-e611-46db-9074-9e930b0588a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:00.039450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff78c199-e611-46db-9074-9e930b0588a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:00.040301 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78c199-e611-46db-9074-9e930b0588a1 None admin] RBAC: Adding query filter params (group.id=75718cd66c97465daa0dfd23189109d5, role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=4677fbdc96dd49fca8554ea63b0d5a7c) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:00.040612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78c199-e611-46db-9074-9e930b0588a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:00.042103 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff78c199-e611-46db-9074-9e930b0588a1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:00.049165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 220/455] 10.222.0.61 () {56 vars in 1376 bytes} [Thu Jul 14 10:00:59 2022] GET /identity/v3/role_assignments?group.id=75718cd66c97465daa0dfd23189109d5&scope.project.id=4677fbdc96dd49fca8554ea63b0d5a7c&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:02.524088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50f6ff-03d8-43bd-9774-4613c08470ad None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:02.524567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50f6ff-03d8-43bd-9774-4613c08470ad None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:02.524567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50f6ff-03d8-43bd-9774-4613c08470ad None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:02.526255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 236/456] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:02.531708 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c410ed-61b4-433a-8e34-d893a4100ab9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:02.532120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c410ed-61b4-433a-8e34-d893a4100ab9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:02.532300 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c410ed-61b4-433a-8e34-d893a4100ab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:02.557914 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c3c410ed-61b4-433a-8e34-d893a4100ab9 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:02.559126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c3c410ed-61b4-433a-8e34-d893a4100ab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:02.647212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 221/457] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:02.684417 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50f6ff-03d8-43bd-9774-4613c08470ad None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:02.711758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:02.818835 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50f6ff-03d8-43bd-9774-4613c08470ad None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:02.862538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00798486-f051-460f-8432-2209977999a1 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:02.862721 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfafcdce-0fca-41d1-892a-b6d61b2b223f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:02.862846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00798486-f051-460f-8432-2209977999a1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:02.863088 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00798486-f051-460f-8432-2209977999a1 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:02.863370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafcdce-0fca-41d1-892a-b6d61b2b223f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:02.863652 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafcdce-0fca-41d1-892a-b6d61b2b223f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:02.863929 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfafcdce-0fca-41d1-892a-b6d61b2b223f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:02.864704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 222/458] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:02.869881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c98c5e-53e4-49b1-88ac-1982bc2afd08 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:02.870781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c98c5e-53e4-49b1-88ac-1982bc2afd08 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:02.871265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c98c5e-53e4-49b1-88ac-1982bc2afd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:02.879404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafcdce-0fca-41d1-892a-b6d61b2b223f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:02.880992 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfafcdce-0fca-41d1-892a-b6d61b2b223f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:02.897859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e3c98c5e-53e4-49b1-88ac-1982bc2afd08 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:02.899359 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e3c98c5e-53e4-49b1-88ac-1982bc2afd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:02.901952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-bfafcdce-0fca-41d1-892a-b6d61b2b223f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:02.901952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:02.901952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:02.901952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:02.901952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:02.901952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:02.901952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:02.901952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:02.901952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:02.901952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:02.901952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:02.901952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:02.901952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:02.901952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:02.901952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:02.902840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:02.903725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:02.903725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:02.903725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:02.903725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:02.903725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 237/459] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:02.905002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fde75-8246-46b1-818d-ab062d4684c9 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:02.908051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fde75-8246-46b1-818d-ab062d4684c9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:02.908051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72fde75-8246-46b1-818d-ab062d4684c9 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:02.908051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 238/460] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:02.909814 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fde75-8246-46b1-818d-ab062d4684c9 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:02.996505 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 223/461] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:02.998300 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816364d8-89f3-400f-9d0d-c4e99961b39f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:02.998731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816364d8-89f3-400f-9d0d-c4e99961b39f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:02.999172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816364d8-89f3-400f-9d0d-c4e99961b39f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:03.019441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:03.020363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-816364d8-89f3-400f-9d0d-c4e99961b39f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:03.021778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-816364d8-89f3-400f-9d0d-c4e99961b39f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:03.067067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72fde75-8246-46b1-818d-ab062d4684c9 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:03.074113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:03.108442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 224/462] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:03.109879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816364d8-89f3-400f-9d0d-c4e99961b39f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:03.115506 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b19772-2483-46fa-9fb3-b7fabc530364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:03.116272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b19772-2483-46fa-9fb3-b7fabc530364 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:03.116775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b19772-2483-46fa-9fb3-b7fabc530364 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:03.117271 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b19772-2483-46fa-9fb3-b7fabc530364 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:03.118285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b19772-2483-46fa-9fb3-b7fabc530364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:03.119552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b19772-2483-46fa-9fb3-b7fabc530364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:03.122063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b19772-2483-46fa-9fb3-b7fabc530364 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:03.139728 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 239/463] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:03.148781 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b19772-2483-46fa-9fb3-b7fabc530364 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:03.244170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816364d8-89f3-400f-9d0d-c4e99961b39f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:03.268241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b19772-2483-46fa-9fb3-b7fabc530364 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:03.289429 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734f74c-aedb-4204-8344-b577fc2ed272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:03.290231 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734f74c-aedb-4204-8344-b577fc2ed272 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:03.290515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734f74c-aedb-4204-8344-b577fc2ed272 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:03.291077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e734f74c-aedb-4204-8344-b577fc2ed272 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:03.304970 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734f74c-aedb-4204-8344-b577fc2ed272 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:03.305230 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3ce59-dd7b-4470-8291-42dbeee2b079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:03.305902 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3ce59-dd7b-4470-8291-42dbeee2b079 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:03.306185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3ce59-dd7b-4470-8291-42dbeee2b079 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:03.306426 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e734f74c-aedb-4204-8344-b577fc2ed272 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:03.306546 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3ce59-dd7b-4470-8291-42dbeee2b079 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:03.307150 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3ce59-dd7b-4470-8291-42dbeee2b079 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:03.308571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3ce59-dd7b-4470-8291-42dbeee2b079 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:03.317146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:03.327346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e734f74c-aedb-4204-8344-b577fc2ed272 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:03.327346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:03.327346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:03.327346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:03.327346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:03.327346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:03.327346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:03.327346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:03.327346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:03.327346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:03.327346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:03.327346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:03.327346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:03.327346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:03.327346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:03.328218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:03.329127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:03.329127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:03.329127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:03.329127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:03.329127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 225/464] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:03.329681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734f74c-aedb-4204-8344-b577fc2ed272 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:03.338529 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 240/465] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:01:03 2022] POST /identity/v3/users => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:01:03.340043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c77dc-35d9-4356-8558-31613bd70eae None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:03.340493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c77dc-35d9-4356-8558-31613bd70eae None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:03.341034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c77dc-35d9-4356-8558-31613bd70eae None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:03.342514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 241/466] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:03.343459 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c77dc-35d9-4356-8558-31613bd70eae None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:03.375733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:03.474997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c77dc-35d9-4356-8558-31613bd70eae None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:03.488728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e734f74c-aedb-4204-8344-b577fc2ed272 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:03.522178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1def6-3a0a-44b7-80d0-d7bb0950f178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:03.522858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1def6-3a0a-44b7-80d0-d7bb0950f178 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:03.523451 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1def6-3a0a-44b7-80d0-d7bb0950f178 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:03.523451 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1def6-3a0a-44b7-80d0-d7bb0950f178 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:03.524126 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d1def6-3a0a-44b7-80d0-d7bb0950f178 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:03.524637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d1def6-3a0a-44b7-80d0-d7bb0950f178 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:03.525820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d1def6-3a0a-44b7-80d0-d7bb0950f178 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:03.538052 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9c332-142f-48eb-a9ef-2631453dec14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:03.538857 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9c332-142f-48eb-a9ef-2631453dec14 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:03.539143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9c332-142f-48eb-a9ef-2631453dec14 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:03.539421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9c332-142f-48eb-a9ef-2631453dec14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:03.542126 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 242/467] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:03.544280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9471c18-286e-49bc-b449-b0016a8abdfb None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:03.544566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9471c18-286e-49bc-b449-b0016a8abdfb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:03.544840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9471c18-286e-49bc-b449-b0016a8abdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:03.553695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9c332-142f-48eb-a9ef-2631453dec14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:03.555221 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9c332-142f-48eb-a9ef-2631453dec14 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:03.566643 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f9471c18-286e-49bc-b449-b0016a8abdfb None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:03.567852 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f9471c18-286e-49bc-b449-b0016a8abdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:03.574569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-b0a9c332-142f-48eb-a9ef-2631453dec14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:03.574569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:03.574569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:03.574569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:03.574569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:03.574569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:03.574569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:03.574569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:03.574569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:03.574569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:03.574569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:03.574569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:03.574569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:03.574569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:03.574569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:03.575465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:03.576363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:03.576363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:03.576363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:03.576363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:03.576363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 226/468] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:03.577711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9c332-142f-48eb-a9ef-2631453dec14 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:03.631732 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:03.654062 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 243/469] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:03.655743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9471c18-286e-49bc-b449-b0016a8abdfb None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:03.685748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:03.711336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9c332-142f-48eb-a9ef-2631453dec14 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:03.762326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e36dbc-5168-4093-a261-24ff1b819394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:03.763565 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e36dbc-5168-4093-a261-24ff1b819394 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:03.764124 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e36dbc-5168-4093-a261-24ff1b819394 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:03.764554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e36dbc-5168-4093-a261-24ff1b819394 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:03.765377 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e36dbc-5168-4093-a261-24ff1b819394 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:03.766605 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9471c18-286e-49bc-b449-b0016a8abdfb None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:03.766958 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e36dbc-5168-4093-a261-24ff1b819394 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:03.797039 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 227/470] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:01:03 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:01:03.798324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e36dbc-5168-4093-a261-24ff1b819394 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:03.828468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380987c-fe7d-4aee-b39f-d37dadc4c858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:03.829611 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380987c-fe7d-4aee-b39f-d37dadc4c858 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:03.832365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380987c-fe7d-4aee-b39f-d37dadc4c858 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:03.833036 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380987c-fe7d-4aee-b39f-d37dadc4c858 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:03.834735 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380987c-fe7d-4aee-b39f-d37dadc4c858 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:03.835985 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380987c-fe7d-4aee-b39f-d37dadc4c858 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:03.839753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380987c-fe7d-4aee-b39f-d37dadc4c858 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:03.868538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 244/471] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:03.874599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380987c-fe7d-4aee-b39f-d37dadc4c858 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:03.950324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:04.010427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e36dbc-5168-4093-a261-24ff1b819394 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:04.018734 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380987c-fe7d-4aee-b39f-d37dadc4c858 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:04.050807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734becd-28ef-4480-aff9-f1336fe2ddc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:04.051986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734becd-28ef-4480-aff9-f1336fe2ddc4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:04.052335 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734becd-28ef-4480-aff9-f1336fe2ddc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:04.052680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734becd-28ef-4480-aff9-f1336fe2ddc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:04.055809 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:04.057019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ef65e-39de-4a1c-86bf-39e28387fcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:04.057696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ef65e-39de-4a1c-86bf-39e28387fcda None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:04.057957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ef65e-39de-4a1c-86bf-39e28387fcda None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:04.058816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ef65e-39de-4a1c-86bf-39e28387fcda None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:04.058951 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ef65e-39de-4a1c-86bf-39e28387fcda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:04.060427 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ef65e-39de-4a1c-86bf-39e28387fcda None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:04.066737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734becd-28ef-4480-aff9-f1336fe2ddc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:04.070230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734becd-28ef-4480-aff9-f1336fe2ddc4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:04.083587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 245/472] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:01:03 2022] POST /identity/v3/users => generated 248 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:01:04.094585 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1ae99-10c5-4fef-8f81-7eb76a3265d7 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:04.094915 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1ae99-10c5-4fef-8f81-7eb76a3265d7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:04.095143 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1ae99-10c5-4fef-8f81-7eb76a3265d7 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:04.096510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 246/473] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:04.097271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-1734becd-28ef-4480-aff9-f1336fe2ddc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:04.097271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:04.097271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:04.097271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:04.097271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:04.097271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:04.097271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:04.097271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:04.097271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:04.097271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:04.097271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:04.097271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:04.097271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:04.097271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:04.097271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:04.098096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:04.098965 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:04.098965 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:04.098965 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:04.098965 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:04.101283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 228/474] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:04.105110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a784ff-e6fb-478c-aebc-360adaae03c1 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:04.105110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a784ff-e6fb-478c-aebc-360adaae03c1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:04.105110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a784ff-e6fb-478c-aebc-360adaae03c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:04.110719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734becd-28ef-4480-aff9-f1336fe2ddc4 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:04.134426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d7a784ff-e6fb-478c-aebc-360adaae03c1 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:04.138296 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d7a784ff-e6fb-478c-aebc-360adaae03c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:04.276490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 247/475] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:04.300671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734becd-28ef-4480-aff9-f1336fe2ddc4 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:04.304975 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a784ff-e6fb-478c-aebc-360adaae03c1 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:04.348575 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:04.355834 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6de69-40db-4e58-ae6d-8f7bfc56ff8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:04.356836 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6de69-40db-4e58-ae6d-8f7bfc56ff8f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:04.357153 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6de69-40db-4e58-ae6d-8f7bfc56ff8f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:04.357432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e6de69-40db-4e58-ae6d-8f7bfc56ff8f None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:04.358168 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6de69-40db-4e58-ae6d-8f7bfc56ff8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:04.358478 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6de69-40db-4e58-ae6d-8f7bfc56ff8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:04.359863 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e6de69-40db-4e58-ae6d-8f7bfc56ff8f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:04.375224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 229/476] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:04.381368 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6de69-40db-4e58-ae6d-8f7bfc56ff8f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:04.451989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:04.454478 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a784ff-e6fb-478c-aebc-360adaae03c1 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:04.510452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77894d1b-aab9-455b-add6-61994e24fdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:04.511919 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77894d1b-aab9-455b-add6-61994e24fdb1 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:04.513036 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77894d1b-aab9-455b-add6-61994e24fdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:04.513734 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77894d1b-aab9-455b-add6-61994e24fdb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:04.514850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77894d1b-aab9-455b-add6-61994e24fdb1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:04.517047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77894d1b-aab9-455b-add6-61994e24fdb1 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:04.522094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e6de69-40db-4e58-ae6d-8f7bfc56ff8f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:04.526304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.sql.core [None req-77894d1b-aab9-455b-add6-61994e24fdb1 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 10:01:04.526304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 10:01:04.526304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [parameters: {'id': '986e055a4a264edc8c155caf634d33fb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 10:01:04.526304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80321) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 10:01:04.548646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-77894d1b-aab9-455b-add6-61994e24fdb1 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 10:01:04.548646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:04.548646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:01:04.548646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:01:04.548646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:01:04.548646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:01:04.548646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:01:04.548646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:01:04.548646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:01:04.548646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:01:04.548646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:01:04.548646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:01:04.548646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:01:04.548646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application result.read() Jul 14 10:01:04.548646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return ref.to_dict() Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:01:04.549719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application yield resource Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise self.value Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application session.commit() Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 10:01:04.550811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self.session.flush() Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application compat.raise_( Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 10:01:04.551850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:01:04.552916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application result.read() Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:01:04.555434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [parameters: {'id': '986e055a4a264edc8c155caf634d33fb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:04.558210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 14 10:01:04.560771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:04.563639 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 248/477] 10.222.0.61 () {60 vars in 1131 bytes} [Thu Jul 14 10:01:04 2022] POST /identity/v3/roles => generated 144 bytes in 251 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:04.566640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77894d1b-aab9-455b-add6-61994e24fdb1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:04.570847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36e574-cff0-4022-8c5c-8b978fe59f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:04.572116 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36e574-cff0-4022-8c5c-8b978fe59f8a None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:04.572589 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36e574-cff0-4022-8c5c-8b978fe59f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:04.573168 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36e574-cff0-4022-8c5c-8b978fe59f8a None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:04.574070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36e574-cff0-4022-8c5c-8b978fe59f8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:04.576372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe36e574-cff0-4022-8c5c-8b978fe59f8a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:04.600179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 230/478] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:01:04 2022] POST /identity/v3/users => generated 249 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:01:04.682182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:04.695235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77894d1b-aab9-455b-add6-61994e24fdb1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:04.739858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e5700-4dda-4608-aad0-8affe19b4323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:04.740744 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e5700-4dda-4608-aad0-8affe19b4323 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:04.741057 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e5700-4dda-4608-aad0-8affe19b4323 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:04.741333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e5700-4dda-4608-aad0-8affe19b4323 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:04.763762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e5700-4dda-4608-aad0-8affe19b4323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-a52e5700-4dda-4608-aad0-8affe19b4323 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:04.772193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:04.773189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:04.774004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:04.774004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:04.774004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:04.774004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:04.774004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:04.774004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:04.774004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:04.774004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:04.774004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:04.774004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:04.774004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 14 10:01:04.774004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:04.778260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 249/479] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:01:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:04.783922 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36e574-cff0-4022-8c5c-8b978fe59f8a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:04.896186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36e574-cff0-4022-8c5c-8b978fe59f8a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:04.898860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:04.932570 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b1204-9ff0-4cfb-9708-587af04b040b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:04.933596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b1204-9ff0-4cfb-9708-587af04b040b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:04.935823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b1204-9ff0-4cfb-9708-587af04b040b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:04.936301 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b1204-9ff0-4cfb-9708-587af04b040b None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:04.937311 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b1204-9ff0-4cfb-9708-587af04b040b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:04.937808 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b1204-9ff0-4cfb-9708-587af04b040b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:04.939656 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66b1204-9ff0-4cfb-9708-587af04b040b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:04.948843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 231/480] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:05.715909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ad15b-883d-485b-aaaf-ca2d05efee98 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:05.716422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ad15b-883d-485b-aaaf-ca2d05efee98 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:05.716498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01ad15b-883d-485b-aaaf-ca2d05efee98 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:05.718129 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 250/481] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:05.723212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125adfef-6c62-47f7-a3d0-0ceb35e94f24 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:05.723623 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125adfef-6c62-47f7-a3d0-0ceb35e94f24 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:05.723792 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125adfef-6c62-47f7-a3d0-0ceb35e94f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:05.746647 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-125adfef-6c62-47f7-a3d0-0ceb35e94f24 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:05.747854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-125adfef-6c62-47f7-a3d0-0ceb35e94f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:05.843393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 232/482] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:05.890829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ad15b-883d-485b-aaaf-ca2d05efee98 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:06.017179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01ad15b-883d-485b-aaaf-ca2d05efee98 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:06.058316 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f9665-1aa4-48af-b6ad-b12f3cdaf44f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:06.060030 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f9665-1aa4-48af-b6ad-b12f3cdaf44f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:06.060531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f9665-1aa4-48af-b6ad-b12f3cdaf44f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:06.061047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f9665-1aa4-48af-b6ad-b12f3cdaf44f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:06.075213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927f9665-1aa4-48af-b6ad-b12f3cdaf44f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-927f9665-1aa4-48af-b6ad-b12f3cdaf44f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:06.083349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:06.084268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:06.085096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:06.085096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:06.085096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:06.085096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:06.085096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:06.085096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:06.085096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:06.085096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:06.085096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:06.085096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:06.085096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:06.085096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:06.087779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 251/483] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:06.093191 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125adfef-6c62-47f7-a3d0-0ceb35e94f24 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:06.221007 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:06.231179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125adfef-6c62-47f7-a3d0-0ceb35e94f24 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:06.266475 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2e535-a481-4b72-ad00-4b57c500127e None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:06.267045 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2e535-a481-4b72-ad00-4b57c500127e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:06.267603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2e535-a481-4b72-ad00-4b57c500127e None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:06.269404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 252/484] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:01:06.273771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d216a8-f8e6-4282-a32e-cef663503bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:06.274439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d216a8-f8e6-4282-a32e-cef663503bdd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:06.274715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d216a8-f8e6-4282-a32e-cef663503bdd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:06.274982 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d216a8-f8e6-4282-a32e-cef663503bdd None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:06.275420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d060253-cc26-44ad-b22a-29d97df8df97 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:06.275732 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d216a8-f8e6-4282-a32e-cef663503bdd None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:06.276034 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d216a8-f8e6-4282-a32e-cef663503bdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:06.276298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d060253-cc26-44ad-b22a-29d97df8df97 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:06.276816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d060253-cc26-44ad-b22a-29d97df8df97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:06.277455 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d216a8-f8e6-4282-a32e-cef663503bdd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:06.286356 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 233/485] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:06.291470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d216a8-f8e6-4282-a32e-cef663503bdd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:06.312204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5d060253-cc26-44ad-b22a-29d97df8df97 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:06.315661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5d060253-cc26-44ad-b22a-29d97df8df97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:06.351404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:06.409270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 253/486] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:01:06.449591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d060253-cc26-44ad-b22a-29d97df8df97 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:06.483398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d216a8-f8e6-4282-a32e-cef663503bdd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:06.543542 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fe8ca-d177-481b-b1cd-b88955c90563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:06.544275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fe8ca-d177-481b-b1cd-b88955c90563 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:06.544651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fe8ca-d177-481b-b1cd-b88955c90563 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:06.544954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919fe8ca-d177-481b-b1cd-b88955c90563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:06.569323 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fe8ca-d177-481b-b1cd-b88955c90563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:06.570847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919fe8ca-d177-481b-b1cd-b88955c90563 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:06.584088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d060253-cc26-44ad-b22a-29d97df8df97 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:06.609145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-919fe8ca-d177-481b-b1cd-b88955c90563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:06.609145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:06.609145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:06.609145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:06.609145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:06.609145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:06.609145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:06.609145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:06.609145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:06.609145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:06.609145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:06.609145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:06.609145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:06.609145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:06.609145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:06.610014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:06.610864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:06.610864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:06.610864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:06.610864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:06.610864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 234/487] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:06.616543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fe8ca-d177-481b-b1cd-b88955c90563 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:06.619562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23283441-7774-4358-87a9-bd51ec53de75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:06.620427 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23283441-7774-4358-87a9-bd51ec53de75 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:06.621334 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23283441-7774-4358-87a9-bd51ec53de75 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:06.621679 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23283441-7774-4358-87a9-bd51ec53de75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:06.632222 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23283441-7774-4358-87a9-bd51ec53de75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:06.638174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-23283441-7774-4358-87a9-bd51ec53de75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:06.640606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:06.641808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:06.642800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:06.642800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:06.642800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:06.642800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:06.642800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:06.642800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:06.642800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:06.642800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:06.642800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:06.642800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:06.642800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:06.642800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:06.645162 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 254/488] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:06.652568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23283441-7774-4358-87a9-bd51ec53de75 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:06.656034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:06.787727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919fe8ca-d177-481b-b1cd-b88955c90563 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:06.841292 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23283441-7774-4358-87a9-bd51ec53de75 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:06.854639 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe3b422-88ed-4ed2-be09-03afd97aadd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:06.855424 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3b422-88ed-4ed2-be09-03afd97aadd1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:06.855706 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3b422-88ed-4ed2-be09-03afd97aadd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:06.855980 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe3b422-88ed-4ed2-be09-03afd97aadd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:06.856785 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3b422-88ed-4ed2-be09-03afd97aadd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:06.857124 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3b422-88ed-4ed2-be09-03afd97aadd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:06.860607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe3b422-88ed-4ed2-be09-03afd97aadd1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:06.880116 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 235/489] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:06.881859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1f728-5b51-4a6e-b7fe-eb6abfc7c207 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:06.882341 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1f728-5b51-4a6e-b7fe-eb6abfc7c207 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:06.882870 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1f728-5b51-4a6e-b7fe-eb6abfc7c207 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:06.884716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 236/490] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:06.885622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1f728-5b51-4a6e-b7fe-eb6abfc7c207 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:06.898101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0aeb730-9593-4e34-9152-cde9199dfe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:06.898506 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aeb730-9593-4e34-9152-cde9199dfe5d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:06.898798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aeb730-9593-4e34-9152-cde9199dfe5d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:06.899075 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aeb730-9593-4e34-9152-cde9199dfe5d None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:06.899898 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aeb730-9593-4e34-9152-cde9199dfe5d None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:06.900215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aeb730-9593-4e34-9152-cde9199dfe5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:06.901734 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0aeb730-9593-4e34-9152-cde9199dfe5d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:06.912231 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 255/491] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:06.913591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58c445-0784-4ec9-a04f-65e298ffad15 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:06.914388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58c445-0784-4ec9-a04f-65e298ffad15 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:06.914692 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58c445-0784-4ec9-a04f-65e298ffad15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:06.935405 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2e58c445-0784-4ec9-a04f-65e298ffad15 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:06.937149 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2e58c445-0784-4ec9-a04f-65e298ffad15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:06.969522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:07.003980 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1f728-5b51-4a6e-b7fe-eb6abfc7c207 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:07.008146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:07.019382 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 256/492] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:07.020606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58c445-0784-4ec9-a04f-65e298ffad15 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:07.063579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f06552-d5f6-4018-a120-30ddd1222513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:07.064385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f06552-d5f6-4018-a120-30ddd1222513 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:07.064665 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f06552-d5f6-4018-a120-30ddd1222513 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:07.064954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f06552-d5f6-4018-a120-30ddd1222513 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:07.072665 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f06552-d5f6-4018-a120-30ddd1222513 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:07.074702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f06552-d5f6-4018-a120-30ddd1222513 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e3f06552-d5f6-4018-a120-30ddd1222513 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:07.091548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:07.092450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:07.093372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:07.093372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:07.093372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:07.093372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:07.093372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:07.093372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:07.093372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:07.093372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:07.093372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:01:07.093372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:07.093372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 237/493] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Jul 14 10:01:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:07.093933 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f06552-d5f6-4018-a120-30ddd1222513 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:07.140352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58c445-0784-4ec9-a04f-65e298ffad15 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:07.188325 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeaa942-4cd5-460c-a39a-9268397bb718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:07.189166 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeaa942-4cd5-460c-a39a-9268397bb718 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:07.189454 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeaa942-4cd5-460c-a39a-9268397bb718 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:07.189738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeaa942-4cd5-460c-a39a-9268397bb718 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:07.202397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeaa942-4cd5-460c-a39a-9268397bb718 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:07.203889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eeaa942-4cd5-460c-a39a-9268397bb718 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:07.257856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-7eeaa942-4cd5-460c-a39a-9268397bb718 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:07.257856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:07.257856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:07.257856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:07.257856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:07.257856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:07.257856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:07.257856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:07.257856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:07.257856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:07.257856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:07.257856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:07.257856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:07.257856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:07.257856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:07.258713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:07.259599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:07.259599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:07.259599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:07.259599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:07.267594 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 257/494] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:07.269858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeaa942-4cd5-460c-a39a-9268397bb718 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:07.315437 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f06552-d5f6-4018-a120-30ddd1222513 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:07.350998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:07.372839 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:07.384333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db68f53-4e3d-4ac3-bda4-a823682750b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:07.385375 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db68f53-4e3d-4ac3-bda4-a823682750b4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:07.385658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db68f53-4e3d-4ac3-bda4-a823682750b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:07.386186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db68f53-4e3d-4ac3-bda4-a823682750b4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:07.410064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db68f53-4e3d-4ac3-bda4-a823682750b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-2db68f53-4e3d-4ac3-bda4-a823682750b4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:07.418050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:07.418939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:07.419740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:07.419740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:07.419740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:07.419740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:07.419740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:07.419740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:07.419740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:07.419740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:07.419740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:07.419740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:07.419740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:07.419740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:07.427921 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 238/495] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:07.429441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db68f53-4e3d-4ac3-bda4-a823682750b4 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:07.452946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eeaa942-4cd5-460c-a39a-9268397bb718 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:07.494289 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298829c-3564-4e38-a7c8-da32560a81bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:07.494964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298829c-3564-4e38-a7c8-da32560a81bb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:07.495337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298829c-3564-4e38-a7c8-da32560a81bb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:07.496144 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298829c-3564-4e38-a7c8-da32560a81bb None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:07.497476 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298829c-3564-4e38-a7c8-da32560a81bb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:07.497797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298829c-3564-4e38-a7c8-da32560a81bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:07.499546 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298829c-3564-4e38-a7c8-da32560a81bb None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:07.538408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 258/496] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Jul 14 10:01:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:07.539670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298829c-3564-4e38-a7c8-da32560a81bb None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:07.557119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db68f53-4e3d-4ac3-bda4-a823682750b4 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:07.594637 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837b927c-fa2c-4d1b-b787-4b2fe0c1f908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:07.595299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b927c-fa2c-4d1b-b787-4b2fe0c1f908 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:07.595582 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b927c-fa2c-4d1b-b787-4b2fe0c1f908 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:07.595858 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837b927c-fa2c-4d1b-b787-4b2fe0c1f908 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:07.596587 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b927c-fa2c-4d1b-b787-4b2fe0c1f908 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:07.596894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b927c-fa2c-4d1b-b787-4b2fe0c1f908 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:07.598306 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837b927c-fa2c-4d1b-b787-4b2fe0c1f908 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:07.610047 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 239/497] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:07.611227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02e2f4-84fc-47d0-a46f-ee40246d41d8 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:07.611512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02e2f4-84fc-47d0-a46f-ee40246d41d8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:07.611782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f02e2f4-84fc-47d0-a46f-ee40246d41d8 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:07.613019 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 240/498] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:07.614074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89887e48-89bf-4095-935c-496f9fd28edc None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:07.614401 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89887e48-89bf-4095-935c-496f9fd28edc None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:07.614874 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89887e48-89bf-4095-935c-496f9fd28edc None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:07.616084 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 241/499] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:07.616777 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89887e48-89bf-4095-935c-496f9fd28edc None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:07.648362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298829c-3564-4e38-a7c8-da32560a81bb None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:07.651039 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:07.671098 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:07.682745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee4012-0996-4a90-9f04-cd080be2ab36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:07.684020 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee4012-0996-4a90-9f04-cd080be2ab36 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:07.684520 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee4012-0996-4a90-9f04-cd080be2ab36 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:07.684798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ee4012-0996-4a90-9f04-cd080be2ab36 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:07.685592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee4012-0996-4a90-9f04-cd080be2ab36 None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:07.685899 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee4012-0996-4a90-9f04-cd080be2ab36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:07.687543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ee4012-0996-4a90-9f04-cd080be2ab36 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:07.695395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 259/500] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:07.696177 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee4012-0996-4a90-9f04-cd080be2ab36 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:07.727133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89887e48-89bf-4095-935c-496f9fd28edc None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:07.761370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ea178-3725-41c7-8e75-99c618cbab5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:07.762015 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ea178-3725-41c7-8e75-99c618cbab5c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:07.762298 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ea178-3725-41c7-8e75-99c618cbab5c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:07.762574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ea178-3725-41c7-8e75-99c618cbab5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:07.793783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ee4012-0996-4a90-9f04-cd080be2ab36 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:07.826553 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84004e8d-cfc1-4ceb-b7f3-dc04ebaa749d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:07.827341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84004e8d-cfc1-4ceb-b7f3-dc04ebaa749d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:07.827624 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84004e8d-cfc1-4ceb-b7f3-dc04ebaa749d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:07.827898 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84004e8d-cfc1-4ceb-b7f3-dc04ebaa749d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:07.828552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ea178-3725-41c7-8e75-99c618cbab5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:07.830060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ea178-3725-41c7-8e75-99c618cbab5c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:07.835318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84004e8d-cfc1-4ceb-b7f3-dc04ebaa749d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:07.836741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84004e8d-cfc1-4ceb-b7f3-dc04ebaa749d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-84004e8d-cfc1-4ceb-b7f3-dc04ebaa749d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:07.851946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:07.853301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:07.854199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:07.854199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:07.854199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:07.854199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:07.854199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:07.854199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:07.854199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:07.854199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:07.854199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:01:07.854199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:07.854199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 260/501] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:01:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:07.855697 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3707c6f-a515-4b7d-a3d9-b652b473e502 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:07.855697 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3707c6f-a515-4b7d-a3d9-b652b473e502 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:07.855828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3707c6f-a515-4b7d-a3d9-b652b473e502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:07.872045 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c3707c6f-a515-4b7d-a3d9-b652b473e502 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:07.873197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c3707c6f-a515-4b7d-a3d9-b652b473e502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:07.880185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:07.892837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:07.929786 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 242/502] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:01:07.931101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8ff24-82f0-42b9-ae41-755b8c78510c None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:07.931386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8ff24-82f0-42b9-ae41-755b8c78510c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:07.931657 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8ff24-82f0-42b9-ae41-755b8c78510c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:07.944202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 261/503] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:07.944991 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3707c6f-a515-4b7d-a3d9-b652b473e502 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:07.949852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b4c8ff24-82f0-42b9-ae41-755b8c78510c None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:07.951043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b4c8ff24-82f0-42b9-ae41-755b8c78510c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:08.018850 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 243/504] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:08.019771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8ff24-82f0-42b9-ae41-755b8c78510c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:08.047320 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3707c6f-a515-4b7d-a3d9-b652b473e502 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:08.081546 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3b93f-59fc-4cec-889e-421bd6021e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:08.082336 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3b93f-59fc-4cec-889e-421bd6021e93 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:08.082615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3b93f-59fc-4cec-889e-421bd6021e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:08.082887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c3b93f-59fc-4cec-889e-421bd6021e93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:08.093430 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3b93f-59fc-4cec-889e-421bd6021e93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:08.094888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c3b93f-59fc-4cec-889e-421bd6021e93 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:08.099729 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:08.113032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-13c3b93f-59fc-4cec-889e-421bd6021e93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:08.113032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:08.113032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:08.113032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:08.113032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:08.113032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:08.113032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:08.113032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:08.113032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:08.113032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:08.113032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:08.113032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:08.113032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:08.113032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:08.113032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:08.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:08.114802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:08.114802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:08.114802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:08.114802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:08.114802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 262/505] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:08.115078 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3b93f-59fc-4cec-889e-421bd6021e93 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:08.123750 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8ff24-82f0-42b9-ae41-755b8c78510c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:08.127805 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:08.157105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c500d23-4a70-4b52-9fda-53984dfbe049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:08.157764 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c500d23-4a70-4b52-9fda-53984dfbe049 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:08.158045 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c500d23-4a70-4b52-9fda-53984dfbe049 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:08.158318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c500d23-4a70-4b52-9fda-53984dfbe049 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:08.159157 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c500d23-4a70-4b52-9fda-53984dfbe049 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:08.159466 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c500d23-4a70-4b52-9fda-53984dfbe049 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:08.160888 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c500d23-4a70-4b52-9fda-53984dfbe049 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:08.197258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 244/506] 10.222.0.61 () {56 vars in 1137 bytes} [Thu Jul 14 10:01:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:08.198152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c500d23-4a70-4b52-9fda-53984dfbe049 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:08.219007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c3b93f-59fc-4cec-889e-421bd6021e93 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:08.254169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c491a2a-da1c-44ba-9cda-32bbe1ce2543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:08.254963 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c491a2a-da1c-44ba-9cda-32bbe1ce2543 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:08.255246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c491a2a-da1c-44ba-9cda-32bbe1ce2543 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:08.255517 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c491a2a-da1c-44ba-9cda-32bbe1ce2543 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:08.267340 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c491a2a-da1c-44ba-9cda-32bbe1ce2543 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:08.268793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c491a2a-da1c-44ba-9cda-32bbe1ce2543 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:08.286032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-4c491a2a-da1c-44ba-9cda-32bbe1ce2543 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:08.286032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:08.286032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:08.286032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:08.286032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:08.286032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:08.286032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:08.286032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:08.286032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:08.286032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:08.286032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:08.286032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:08.286032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:08.286032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:08.286032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:08.287032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:08.288073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:08.288073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:08.288073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:08.288073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:08.288073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 263/507] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:08.288073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c491a2a-da1c-44ba-9cda-32bbe1ce2543 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:08.299823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c500d23-4a70-4b52-9fda-53984dfbe049 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:08.332867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80734b4b-c6b2-402b-a7de-0647df8aff28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:08.333671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80734b4b-c6b2-402b-a7de-0647df8aff28 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:08.333948 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80734b4b-c6b2-402b-a7de-0647df8aff28 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:08.334212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80734b4b-c6b2-402b-a7de-0647df8aff28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:08.341404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:08.345350 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80734b4b-c6b2-402b-a7de-0647df8aff28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:08.348756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-80734b4b-c6b2-402b-a7de-0647df8aff28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:08.353193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:08.354105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:08.355107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:08.355107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:08.355107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:08.355107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:08.355107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:08.355107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:08.355107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:08.355107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:08.355107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:08.355107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:08.355107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:08.355107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:08.355107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 245/508] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:08.355778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80734b4b-c6b2-402b-a7de-0647df8aff28 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:08.395077 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c491a2a-da1c-44ba-9cda-32bbe1ce2543 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:08.427245 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667384a7-380f-491d-ab75-643604dab730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:08.428031 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667384a7-380f-491d-ab75-643604dab730 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:08.428311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667384a7-380f-491d-ab75-643604dab730 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:08.428583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667384a7-380f-491d-ab75-643604dab730 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:08.439251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667384a7-380f-491d-ab75-643604dab730 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:08.440688 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667384a7-380f-491d-ab75-643604dab730 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:08.454781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80734b4b-c6b2-402b-a7de-0647df8aff28 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:08.458367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-667384a7-380f-491d-ab75-643604dab730 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:08.458367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:08.458367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:08.458367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:08.458367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:08.458367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:08.458367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:08.458367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:08.458367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:08.458367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:08.458367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:08.458367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:08.458367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:08.458367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:08.458367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:08.459321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:08.460181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:08.460181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:08.460181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:08.460181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:08.460181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 264/509] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:08.460450 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667384a7-380f-491d-ab75-643604dab730 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:08.487798 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938f2ea-853b-483f-8b93-c129c4c3312a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:08.488459 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938f2ea-853b-483f-8b93-c129c4c3312a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:08.488741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938f2ea-853b-483f-8b93-c129c4c3312a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:08.489063 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938f2ea-853b-483f-8b93-c129c4c3312a None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:08.489790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938f2ea-853b-483f-8b93-c129c4c3312a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:08.490097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938f2ea-853b-483f-8b93-c129c4c3312a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:08.491508 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938f2ea-853b-483f-8b93-c129c4c3312a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:08.503688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 246/510] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:08.504482 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938f2ea-853b-483f-8b93-c129c4c3312a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:08.560043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667384a7-380f-491d-ab75-643604dab730 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:08.562342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:08.574090 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:08.594750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cc457-5a65-4963-963a-893ac8dd5ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:08.595398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cc457-5a65-4963-963a-893ac8dd5ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:08.595680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cc457-5a65-4963-963a-893ac8dd5ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:08.595953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1cc457-5a65-4963-963a-893ac8dd5ce0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:08.602509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938f2ea-853b-483f-8b93-c129c4c3312a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:08.639029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3e319-bd51-47a1-b79a-620e124eb1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:08.639681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3e319-bd51-47a1-b79a-620e124eb1aa None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:08.639963 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3e319-bd51-47a1-b79a-620e124eb1aa None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:08.640235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e3e319-bd51-47a1-b79a-620e124eb1aa None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:08.640983 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3e319-bd51-47a1-b79a-620e124eb1aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:08.641293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3e319-bd51-47a1-b79a-620e124eb1aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:08.642659 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e3e319-bd51-47a1-b79a-620e124eb1aa None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:08.658538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 247/511] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:08.658538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3e319-bd51-47a1-b79a-620e124eb1aa None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:08.662797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1cc457-5a65-4963-963a-893ac8dd5ce0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:08.664243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1cc457-5a65-4963-963a-893ac8dd5ce0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:08.755883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e3e319-bd51-47a1-b79a-620e124eb1aa None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:08.760507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 265/512] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:01:08.761937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cc457-5a65-4963-963a-893ac8dd5ce0 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:08.775365 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:08.776965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:08.793893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b290f-a1bf-4ab3-b626-ceeb014303d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:08.794557 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b290f-a1bf-4ab3-b626-ceeb014303d5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:08.794839 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b290f-a1bf-4ab3-b626-ceeb014303d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:08.795115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b290f-a1bf-4ab3-b626-ceeb014303d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:08.795856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b290f-a1bf-4ab3-b626-ceeb014303d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:08.796163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b290f-a1bf-4ab3-b626-ceeb014303d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:08.797559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b290f-a1bf-4ab3-b626-ceeb014303d5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:08.805318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 248/513] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:08.806098 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b290f-a1bf-4ab3-b626-ceeb014303d5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:08.862238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1cc457-5a65-4963-963a-893ac8dd5ce0 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:08.895388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c7cde-c3b3-4fff-a17c-6d853f50e3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:08.896044 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c7cde-c3b3-4fff-a17c-6d853f50e3cc None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:08.896323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c7cde-c3b3-4fff-a17c-6d853f50e3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:08.896597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c7cde-c3b3-4fff-a17c-6d853f50e3cc None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:08.897346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c7cde-c3b3-4fff-a17c-6d853f50e3cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:08.897650 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c7cde-c3b3-4fff-a17c-6d853f50e3cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:08.899001 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c7cde-c3b3-4fff-a17c-6d853f50e3cc None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:08.903216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b290f-a1bf-4ab3-b626-ceeb014303d5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:08.909815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 266/514] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:08.910562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c7cde-c3b3-4fff-a17c-6d853f50e3cc None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:08.941624 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc7ac4-0082-433a-b07d-87fcfddb6e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:08.942440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc7ac4-0082-433a-b07d-87fcfddb6e7a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:08.942722 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc7ac4-0082-433a-b07d-87fcfddb6e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:08.942994 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc7ac4-0082-433a-b07d-87fcfddb6e7a None admin] PATH_INFO: `/v3/users/glance` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:08.950941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc7ac4-0082-433a-b07d-87fcfddb6e7a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:08.952362 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc7ac4-0082-433a-b07d-87fcfddb6e7a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-0efc7ac4-0082-433a-b07d-87fcfddb6e7a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:08.966275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:08.967202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:08.968281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:08.968281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:08.968281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:08.968281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:08.968281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:08.968281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:08.968281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:08.968281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:08.968281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:01:08.968281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:08.968281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 249/515] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:01:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:08.968281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc7ac4-0082-433a-b07d-87fcfddb6e7a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:08.981524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:09.009740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:09.012738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c7cde-c3b3-4fff-a17c-6d853f50e3cc None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:09.045966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a0e7d-92c2-4ffa-9de5-346d0008a0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:09.046751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a0e7d-92c2-4ffa-9de5-346d0008a0a5 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:09.047026 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a0e7d-92c2-4ffa-9de5-346d0008a0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:09.047301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a0e7d-92c2-4ffa-9de5-346d0008a0a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:09.053826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a0e7d-92c2-4ffa-9de5-346d0008a0a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:09.055262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a0e7d-92c2-4ffa-9de5-346d0008a0a5 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:09.068310 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc7ac4-0082-433a-b07d-87fcfddb6e7a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:09.071063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-3c2a0e7d-92c2-4ffa-9de5-346d0008a0a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:09.071063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:09.071063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:09.071063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:09.071063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:09.071063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:09.071063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:09.071063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:09.071063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:09.071063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:09.071063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:09.071063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:09.071063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:09.071063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:09.071063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:09.071917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:09.072778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:09.072778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:09.072778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:09.072778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:09.072778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:09.072778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:09.072778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:09.072778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:09.072778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 267/516] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:09.074225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a0e7d-92c2-4ffa-9de5-346d0008a0a5 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:09.101642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a985f-ee95-4bc5-99a4-928b2200200d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:09.102430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a985f-ee95-4bc5-99a4-928b2200200d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:09.102713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a985f-ee95-4bc5-99a4-928b2200200d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:09.102991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a985f-ee95-4bc5-99a4-928b2200200d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:09.113425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a985f-ee95-4bc5-99a4-928b2200200d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:09.114867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a985f-ee95-4bc5-99a4-928b2200200d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:09.132119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-b80a985f-ee95-4bc5-99a4-928b2200200d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:09.132119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:09.132119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:09.132119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:09.132119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:09.132119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:09.132119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:09.132119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:09.132119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:09.132119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:09.132119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:09.132119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:09.132119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:09.132119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:09.132119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:09.133013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:09.133861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:09.133861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:09.133861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:09.133861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:09.133861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 250/517] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:09.134133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a985f-ee95-4bc5-99a4-928b2200200d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:09.176470 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a0e7d-92c2-4ffa-9de5-346d0008a0a5 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:09.205934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:09.211492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b886f3-af24-486e-b3c3-650d869e2d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:09.212276 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b886f3-af24-486e-b3c3-650d869e2d59 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:09.212556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b886f3-af24-486e-b3c3-650d869e2d59 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:09.212824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b886f3-af24-486e-b3c3-650d869e2d59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:09.222602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b886f3-af24-486e-b3c3-650d869e2d59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:09.223575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:09.224039 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b886f3-af24-486e-b3c3-650d869e2d59 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:09.233182 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a985f-ee95-4bc5-99a4-928b2200200d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:09.241654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-51b886f3-af24-486e-b3c3-650d869e2d59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:09.241654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:09.241654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:09.241654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:09.241654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:09.241654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:09.241654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:09.241654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:09.241654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:09.241654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:09.241654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:09.241654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:09.241654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:09.241654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:09.241654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:09.242547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:09.244095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:09.244095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:09.244095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:09.244095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:09.244095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 268/518] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:09.244095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b886f3-af24-486e-b3c3-650d869e2d59 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:09.271655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e165c1-2bd0-4d12-9c67-0ddc46c45bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:09.272321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e165c1-2bd0-4d12-9c67-0ddc46c45bf9 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:09.272599 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e165c1-2bd0-4d12-9c67-0ddc46c45bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:09.272868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e165c1-2bd0-4d12-9c67-0ddc46c45bf9 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:09.273600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e165c1-2bd0-4d12-9c67-0ddc46c45bf9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:09.275038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e165c1-2bd0-4d12-9c67-0ddc46c45bf9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:09.295416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 251/519] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:01:09 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:01:09.296037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e165c1-2bd0-4d12-9c67-0ddc46c45bf9 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:09.361350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b886f3-af24-486e-b3c3-650d869e2d59 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:09.396844 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6eec8-6d84-4d99-b889-42059b8e875a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:09.397554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6eec8-6d84-4d99-b889-42059b8e875a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:09.397836 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6eec8-6d84-4d99-b889-42059b8e875a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:09.398103 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa6eec8-6d84-4d99-b889-42059b8e875a None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:09.398950 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6eec8-6d84-4d99-b889-42059b8e875a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:09.399263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6eec8-6d84-4d99-b889-42059b8e875a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:09.400698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa6eec8-6d84-4d99-b889-42059b8e875a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:09.409963 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e165c1-2bd0-4d12-9c67-0ddc46c45bf9 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:09.424355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 269/520] 10.222.0.61 () {56 vars in 1137 bytes} [Thu Jul 14 10:01:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:09.424355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6eec8-6d84-4d99-b889-42059b8e875a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:09.454333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26c00d-eb0e-4976-9a82-6ef5ca16389a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:09.455152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26c00d-eb0e-4976-9a82-6ef5ca16389a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:09.455434 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26c00d-eb0e-4976-9a82-6ef5ca16389a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:09.455888 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff26c00d-eb0e-4976-9a82-6ef5ca16389a None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:09.456692 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26c00d-eb0e-4976-9a82-6ef5ca16389a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:09.457035 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26c00d-eb0e-4976-9a82-6ef5ca16389a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:09.458545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff26c00d-eb0e-4976-9a82-6ef5ca16389a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:09.468981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:09.477739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 252/521] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:09.478592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26c00d-eb0e-4976-9a82-6ef5ca16389a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:09.491983 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:09.542405 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa6eec8-6d84-4d99-b889-42059b8e875a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:09.577456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83c948-2fbd-47ad-af82-dcb2d9d44c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:09.578118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83c948-2fbd-47ad-af82-dcb2d9d44c1a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:09.578397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83c948-2fbd-47ad-af82-dcb2d9d44c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:09.578676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b83c948-2fbd-47ad-af82-dcb2d9d44c1a None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:09.579394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83c948-2fbd-47ad-af82-dcb2d9d44c1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:09.579702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83c948-2fbd-47ad-af82-dcb2d9d44c1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:09.581091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b83c948-2fbd-47ad-af82-dcb2d9d44c1a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:09.597367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 270/522] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:09.597367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83c948-2fbd-47ad-af82-dcb2d9d44c1a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:09.597634 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff26c00d-eb0e-4976-9a82-6ef5ca16389a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:09.634950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a702a-9dc2-4741-85c7-c9c7dec76bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:09.635803 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a702a-9dc2-4741-85c7-c9c7dec76bcf None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:09.636162 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a702a-9dc2-4741-85c7-c9c7dec76bcf None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:09.636510 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a702a-9dc2-4741-85c7-c9c7dec76bcf None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:09.637350 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a702a-9dc2-4741-85c7-c9c7dec76bcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:09.637760 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a702a-9dc2-4741-85c7-c9c7dec76bcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:09.639231 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a702a-9dc2-4741-85c7-c9c7dec76bcf None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:09.652783 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 253/523] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:09.653662 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a702a-9dc2-4741-85c7-c9c7dec76bcf None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:09.710679 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b83c948-2fbd-47ad-af82-dcb2d9d44c1a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:09.712313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:09.723819 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:09.749906 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92d154-0ef5-407a-9ab6-8ee49daa89ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:09.750714 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92d154-0ef5-407a-9ab6-8ee49daa89ec None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:09.751097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92d154-0ef5-407a-9ab6-8ee49daa89ec None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:09.751466 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92d154-0ef5-407a-9ab6-8ee49daa89ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:09.760636 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a702a-9dc2-4741-85c7-c9c7dec76bcf None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:09.796790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b78c3-8b06-45d7-a9c2-357005b306b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:09.799121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b78c3-8b06-45d7-a9c2-357005b306b4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:09.799121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b78c3-8b06-45d7-a9c2-357005b306b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:09.799121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7b78c3-8b06-45d7-a9c2-357005b306b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:09.799121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b78c3-8b06-45d7-a9c2-357005b306b4 None admin] RBAC: Adding query filter params (role.id=ba4088a289ea4b9a9aa9d9cd7a27e731, scope.project.id=57662256880f43018039715a755c087c, user.id=e9f4efa3e2f343c2ac858a277601a7df) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:09.799121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b78c3-8b06-45d7-a9c2-357005b306b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:09.800569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7b78c3-8b06-45d7-a9c2-357005b306b4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:09.806813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 254/524] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:09 2022] GET /identity/v3/role_assignments?user.id=e9f4efa3e2f343c2ac858a277601a7df&scope.project.id=57662256880f43018039715a755c087c&role.id=ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:09.806813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b78c3-8b06-45d7-a9c2-357005b306b4 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:09.836496 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92d154-0ef5-407a-9ab6-8ee49daa89ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:09.839453 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92d154-0ef5-407a-9ab6-8ee49daa89ec None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:09.964468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7b78c3-8b06-45d7-a9c2-357005b306b4 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:09.965296 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 271/525] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:01:09.966222 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92d154-0ef5-407a-9ab6-8ee49daa89ec None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:09.971136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:10.008679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:10.014772 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07a5c4-8e96-46f2-b1ed-c1c36dcc9d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:10.016116 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07a5c4-8e96-46f2-b1ed-c1c36dcc9d08 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:10.016411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07a5c4-8e96-46f2-b1ed-c1c36dcc9d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:10.016688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d07a5c4-8e96-46f2-b1ed-c1c36dcc9d08 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:10.026882 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07a5c4-8e96-46f2-b1ed-c1c36dcc9d08 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:10.028683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d07a5c4-8e96-46f2-b1ed-c1c36dcc9d08 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:10.048607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-2d07a5c4-8e96-46f2-b1ed-c1c36dcc9d08 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:10.048607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:10.048607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:10.048607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:10.048607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:10.048607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:10.048607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:10.048607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:10.048607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:10.048607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:10.048607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:10.048607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:10.048607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:10.048607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:10.048607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:10.049524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:10.050374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:10.050374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:10.050374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:10.050374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:10.050374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:10.050374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:10.050374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:10.050374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:10.054806 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 255/526] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:10.055672 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07a5c4-8e96-46f2-b1ed-c1c36dcc9d08 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:10.085280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92d154-0ef5-407a-9ab6-8ee49daa89ec None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:10.124885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bd6b2-af70-4b6e-b6e4-c9a182799d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:10.125820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bd6b2-af70-4b6e-b6e4-c9a182799d24 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:10.126197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bd6b2-af70-4b6e-b6e4-c9a182799d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:10.126540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bd6b2-af70-4b6e-b6e4-c9a182799d24 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:10.134340 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bd6b2-af70-4b6e-b6e4-c9a182799d24 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:10.137202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bd6b2-af70-4b6e-b6e4-c9a182799d24 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-838bd6b2-af70-4b6e-b6e4-c9a182799d24 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:10.153395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:10.154321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:10.155441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:10.155441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:10.155441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:10.155441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:10.155441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:10.155441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:10.155441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:10.155441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:10.155441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:01:10.155441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:10.155441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 272/527] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:10.156402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bd6b2-af70-4b6e-b6e4-c9a182799d24 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:10.180599 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d07a5c4-8e96-46f2-b1ed-c1c36dcc9d08 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:10.220230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f860c-f760-4fe6-9a78-2535a837975a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:10.220964 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f860c-f760-4fe6-9a78-2535a837975a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:10.221252 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f860c-f760-4fe6-9a78-2535a837975a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:10.221524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f860c-f760-4fe6-9a78-2535a837975a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:10.222543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:10.233351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f860c-f760-4fe6-9a78-2535a837975a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:10.234989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f860c-f760-4fe6-9a78-2535a837975a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:10.254170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-1f7f860c-f760-4fe6-9a78-2535a837975a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:10.254170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:10.254170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:10.254170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:10.254170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:10.254170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:10.254170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:10.254170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:10.254170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:10.254170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:10.254170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:10.254170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:10.254170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:10.254170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:10.254170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:10.255127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:10.255989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:10.255989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:10.255989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:10.255989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:10.257779 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 256/528] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:10.258772 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f860c-f760-4fe6-9a78-2535a837975a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:10.266922 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bd6b2-af70-4b6e-b6e4-c9a182799d24 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:10.302050 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c08f88-29a6-45ab-8256-6e25b1dcbb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:10.302512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c08f88-29a6-45ab-8256-6e25b1dcbb25 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:10.302798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c08f88-29a6-45ab-8256-6e25b1dcbb25 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:10.303074 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c08f88-29a6-45ab-8256-6e25b1dcbb25 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:10.303866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c08f88-29a6-45ab-8256-6e25b1dcbb25 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:10.304977 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c08f88-29a6-45ab-8256-6e25b1dcbb25 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:10.305960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c08f88-29a6-45ab-8256-6e25b1dcbb25 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:10.310982 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:10.336091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 273/529] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:10.337497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c08f88-29a6-45ab-8256-6e25b1dcbb25 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:10.382830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f860c-f760-4fe6-9a78-2535a837975a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:10.430048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04377f0c-b188-4ce3-a976-83afa7171f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:10.430844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04377f0c-b188-4ce3-a976-83afa7171f78 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:10.431191 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04377f0c-b188-4ce3-a976-83afa7171f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:10.431533 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04377f0c-b188-4ce3-a976-83afa7171f78 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:10.433020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04377f0c-b188-4ce3-a976-83afa7171f78 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:10.433938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04377f0c-b188-4ce3-a976-83afa7171f78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:10.435923 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04377f0c-b188-4ce3-a976-83afa7171f78 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:10.466981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 257/530] 10.222.0.61 () {56 vars in 1139 bytes} [Thu Jul 14 10:01:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:10.466981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04377f0c-b188-4ce3-a976-83afa7171f78 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:10.467847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c08f88-29a6-45ab-8256-6e25b1dcbb25 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:10.492446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:10.513135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185da29b-863a-4b0f-8d1a-dc260a582238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:10.513849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185da29b-863a-4b0f-8d1a-dc260a582238 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:10.514129 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185da29b-863a-4b0f-8d1a-dc260a582238 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:10.514396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185da29b-863a-4b0f-8d1a-dc260a582238 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:10.515120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185da29b-863a-4b0f-8d1a-dc260a582238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:10.515422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185da29b-863a-4b0f-8d1a-dc260a582238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:10.516793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185da29b-863a-4b0f-8d1a-dc260a582238 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:10.533273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 274/531] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:10.534526 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185da29b-863a-4b0f-8d1a-dc260a582238 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:10.584732 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04377f0c-b188-4ce3-a976-83afa7171f78 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:10.587144 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:10.634645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2585a9-efc6-4863-800c-3d6187e2f4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:10.636793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2585a9-efc6-4863-800c-3d6187e2f4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:10.637268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2585a9-efc6-4863-800c-3d6187e2f4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:10.638009 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2585a9-efc6-4863-800c-3d6187e2f4b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:10.639257 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2585a9-efc6-4863-800c-3d6187e2f4b1 None admin] RBAC: Adding query filter params (role.id=ba4088a289ea4b9a9aa9d9cd7a27e731, scope.project.id=57662256880f43018039715a755c087c, user.id=52e3fbd0400a49a69932ce426218f4a6) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:10.639749 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2585a9-efc6-4863-800c-3d6187e2f4b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:10.641709 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2585a9-efc6-4863-800c-3d6187e2f4b1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:10.650138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 258/532] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:10 2022] GET /identity/v3/role_assignments?user.id=52e3fbd0400a49a69932ce426218f4a6&scope.project.id=57662256880f43018039715a755c087c&role.id=ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:10.651965 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2585a9-efc6-4863-800c-3d6187e2f4b1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:10.656203 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185da29b-863a-4b0f-8d1a-dc260a582238 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:10.693994 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace2191-06e4-4ef9-8f10-c69c7b79e133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:10.694747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2191-06e4-4ef9-8f10-c69c7b79e133 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:10.695179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2191-06e4-4ef9-8f10-c69c7b79e133 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:10.695419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2191-06e4-4ef9-8f10-c69c7b79e133 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:10.804217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:10.839461 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace2191-06e4-4ef9-8f10-c69c7b79e133 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:10.840937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace2191-06e4-4ef9-8f10-c69c7b79e133 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:10.851045 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2585a9-efc6-4863-800c-3d6187e2f4b1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:10.893847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db35015-b49d-4bdd-819a-878518670f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:10.894980 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db35015-b49d-4bdd-819a-878518670f44 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:10.895467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db35015-b49d-4bdd-819a-878518670f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:10.895938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db35015-b49d-4bdd-819a-878518670f44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:10.910160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db35015-b49d-4bdd-819a-878518670f44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:10.913322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db35015-b49d-4bdd-819a-878518670f44 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:10.938373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-9db35015-b49d-4bdd-819a-878518670f44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:10.938373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:10.938373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:10.938373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:10.938373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:10.938373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:10.938373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:10.938373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:10.938373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:10.938373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:10.938373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:10.938373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:10.938373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:10.938373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:10.938373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:10.939261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:10.940111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:10.940111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:10.940111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:10.940111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:10.940111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:10.940111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:10.940111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:10.940111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:10.952719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 259/533] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:10.960405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db35015-b49d-4bdd-819a-878518670f44 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:10.981208 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 275/534] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:01:10.988855 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace2191-06e4-4ef9-8f10-c69c7b79e133 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:11.018174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:11.153283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:11.167880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db35015-b49d-4bdd-819a-878518670f44 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:11.185785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace2191-06e4-4ef9-8f10-c69c7b79e133 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:11.215503 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc0041-ab12-4684-827a-932c39754e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:11.216180 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc0041-ab12-4684-827a-932c39754e28 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:11.216461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc0041-ab12-4684-827a-932c39754e28 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:11.216735 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fc0041-ab12-4684-827a-932c39754e28 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:11.217558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc0041-ab12-4684-827a-932c39754e28 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:11.217871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc0041-ab12-4684-827a-932c39754e28 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:11.219309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fc0041-ab12-4684-827a-932c39754e28 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:11.231150 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e67edb-8770-43e6-a933-57e0639079b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:11.231943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e67edb-8770-43e6-a933-57e0639079b6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:11.232228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e67edb-8770-43e6-a933-57e0639079b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:11.232424 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 260/535] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:11.232524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e67edb-8770-43e6-a933-57e0639079b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:11.241921 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc0041-ab12-4684-827a-932c39754e28 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:11.243393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e67edb-8770-43e6-a933-57e0639079b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:11.244963 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e67edb-8770-43e6-a933-57e0639079b6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:11.265072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-90e67edb-8770-43e6-a933-57e0639079b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:11.265072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:11.265072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:11.265072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:11.265072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:11.265072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:11.265072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:11.265072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:11.265072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:11.265072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:11.265072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:11.265072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:11.265072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:11.265072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:11.265072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:11.265966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:11.266824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:11.266824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:11.266824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:11.266824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:11.266824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 276/536] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:11.278021 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e67edb-8770-43e6-a933-57e0639079b6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:11.345368 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:11.382353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fc0041-ab12-4684-827a-932c39754e28 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:11.443904 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c7f1ab-ecb1-4d47-9c0a-3441b06541e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:11.444479 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7f1ab-ecb1-4d47-9c0a-3441b06541e6 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:11.444759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7f1ab-ecb1-4d47-9c0a-3441b06541e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:11.445061 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c7f1ab-ecb1-4d47-9c0a-3441b06541e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:11.445898 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7f1ab-ecb1-4d47-9c0a-3441b06541e6 None admin] RBAC: Adding query filter params (role.id=ba4088a289ea4b9a9aa9d9cd7a27e731, scope.project.id=57662256880f43018039715a755c087c, user.id=e3b5660cfd344dcd8abe14c88df686e8) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:11.446211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7f1ab-ecb1-4d47-9c0a-3441b06541e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:11.448182 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c7f1ab-ecb1-4d47-9c0a-3441b06541e6 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:11.454195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 261/537] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:11 2022] GET /identity/v3/role_assignments?user.id=e3b5660cfd344dcd8abe14c88df686e8&scope.project.id=57662256880f43018039715a755c087c&role.id=ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:11.455440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667db87-0b34-407e-9a3f-e5845ba8eba8 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:11.455729 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667db87-0b34-407e-9a3f-e5845ba8eba8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:11.456006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667db87-0b34-407e-9a3f-e5845ba8eba8 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:11.465188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e67edb-8770-43e6-a933-57e0639079b6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:11.465831 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 262/538] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:11 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:11.471636 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3db38-1f79-440c-a2fb-a610214ab5da None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:11.471926 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3db38-1f79-440c-a2fb-a610214ab5da None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:11.472155 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b3db38-1f79-440c-a2fb-a610214ab5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:11.504531 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-81b3db38-1f79-440c-a2fb-a610214ab5da None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:11.508555 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-81b3db38-1f79-440c-a2fb-a610214ab5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:11.520409 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77e66c-0017-4f89-a212-65b8ff022742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:11.523978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77e66c-0017-4f89-a212-65b8ff022742 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:11.524569 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77e66c-0017-4f89-a212-65b8ff022742 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:11.525122 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77e66c-0017-4f89-a212-65b8ff022742 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:11.526708 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77e66c-0017-4f89-a212-65b8ff022742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:11.527265 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77e66c-0017-4f89-a212-65b8ff022742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:11.529194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77e66c-0017-4f89-a212-65b8ff022742 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:11.531157 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:11.558577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 277/539] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:11.564580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77e66c-0017-4f89-a212-65b8ff022742 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:11.668679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 263/540] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:01:11.697832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3db38-1f79-440c-a2fb-a610214ab5da None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:11.735530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77e66c-0017-4f89-a212-65b8ff022742 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:11.777512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23705aa4-76db-40b7-a037-139d39e44e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:11.778316 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23705aa4-76db-40b7-a037-139d39e44e68 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:11.778601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23705aa4-76db-40b7-a037-139d39e44e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:11.779417 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23705aa4-76db-40b7-a037-139d39e44e68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:11.786501 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23705aa4-76db-40b7-a037-139d39e44e68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:11.788871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23705aa4-76db-40b7-a037-139d39e44e68 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:11.805818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-23705aa4-76db-40b7-a037-139d39e44e68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:11.805818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:11.805818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:11.805818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:11.805818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:11.805818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:11.805818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:11.805818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:11.805818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:11.805818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:11.805818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:11.805818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:11.805818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:11.805818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:11.805818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:11.806752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:11.807592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:11.807592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:11.807592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:11.807592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:11.807592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:11.807592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:11.807592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:11.807592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:11.808008 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 278/541] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:11.813753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23705aa4-76db-40b7-a037-139d39e44e68 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:11.877777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:11.883131 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:11.932288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23705aa4-76db-40b7-a037-139d39e44e68 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:11.935308 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b3db38-1f79-440c-a2fb-a610214ab5da None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:11.973395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4346441-df77-412b-b876-03511b9c002a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:11.974005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4346441-df77-412b-b876-03511b9c002a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:11.974286 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4346441-df77-412b-b876-03511b9c002a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:11.974563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4346441-df77-412b-b876-03511b9c002a None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:11.975730 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4346441-df77-412b-b876-03511b9c002a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:11.976069 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4346441-df77-412b-b876-03511b9c002a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:11.978305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4346441-df77-412b-b876-03511b9c002a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:11.993327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 279/542] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:12.001467 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4346441-df77-412b-b876-03511b9c002a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:12.022551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef33214-cbdd-446b-a4a0-7538fd0c1b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:12.023357 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef33214-cbdd-446b-a4a0-7538fd0c1b3b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:12.023634 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef33214-cbdd-446b-a4a0-7538fd0c1b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:12.023904 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef33214-cbdd-446b-a4a0-7538fd0c1b3b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:12.047744 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef33214-cbdd-446b-a4a0-7538fd0c1b3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-8ef33214-cbdd-446b-a4a0-7538fd0c1b3b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:12.063846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:12.065941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:12.067844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:12.067844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:12.067844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:12.067844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:12.067844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:12.067844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:12.067844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:12.067844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:12.067844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:12.067844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:12.067844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:12.067844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:12.067844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 264/543] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:12.077904 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef33214-cbdd-446b-a4a0-7538fd0c1b3b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:12.113660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4346441-df77-412b-b876-03511b9c002a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:12.137373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:12.148120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e837b9-6789-4ba3-b06a-63a6bd259309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:12.148844 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e837b9-6789-4ba3-b06a-63a6bd259309 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:12.151797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e837b9-6789-4ba3-b06a-63a6bd259309 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:12.152081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e837b9-6789-4ba3-b06a-63a6bd259309 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:12.152940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e837b9-6789-4ba3-b06a-63a6bd259309 None admin] RBAC: Adding query filter params (role.id=ba4088a289ea4b9a9aa9d9cd7a27e731, scope.project.id=57662256880f43018039715a755c087c, user.id=cd6d19c4406b4b228065b98a9a074b3a) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:12.153261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e837b9-6789-4ba3-b06a-63a6bd259309 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:12.154696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e837b9-6789-4ba3-b06a-63a6bd259309 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:12.160745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 280/544] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:12 2022] GET /identity/v3/role_assignments?user.id=cd6d19c4406b4b228065b98a9a074b3a&scope.project.id=57662256880f43018039715a755c087c&role.id=ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:12.162198 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6296e4f-fd0d-49d7-835e-5fb5636c40cb None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:12.162488 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6296e4f-fd0d-49d7-835e-5fb5636c40cb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:12.162767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6296e4f-fd0d-49d7-835e-5fb5636c40cb None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:12.164370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 281/545] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:12.171425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ac496-799b-4348-bf2e-be2659c33549 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:12.171717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ac496-799b-4348-bf2e-be2659c33549 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:12.171998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ac496-799b-4348-bf2e-be2659c33549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:12.190572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-135ac496-799b-4348-bf2e-be2659c33549 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:12.191879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-135ac496-799b-4348-bf2e-be2659c33549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:12.251399 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef33214-cbdd-446b-a4a0-7538fd0c1b3b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:12.320333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717ce88-aec1-4269-83dd-0bc75d4476a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:12.321053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717ce88-aec1-4269-83dd-0bc75d4476a5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:12.321339 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717ce88-aec1-4269-83dd-0bc75d4476a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:12.329114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 282/546] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:12.329558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e717ce88-aec1-4269-83dd-0bc75d4476a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:12.330501 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717ce88-aec1-4269-83dd-0bc75d4476a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:12.330823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717ce88-aec1-4269-83dd-0bc75d4476a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:12.332266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e717ce88-aec1-4269-83dd-0bc75d4476a5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:12.342555 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 265/547] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:12.347855 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ac496-799b-4348-bf2e-be2659c33549 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:12.367361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717ce88-aec1-4269-83dd-0bc75d4476a5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:12.372345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:12.493140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ac496-799b-4348-bf2e-be2659c33549 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:12.508389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:12.518118 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e717ce88-aec1-4269-83dd-0bc75d4476a5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:12.565630 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc12e09-66e6-400b-9497-246ae3d52b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:12.566138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94868061-ef79-4af3-910b-a71f19076313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:12.566445 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc12e09-66e6-400b-9497-246ae3d52b33 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:12.566729 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc12e09-66e6-400b-9497-246ae3d52b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:12.566998 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94868061-ef79-4af3-910b-a71f19076313 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:12.567163 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc12e09-66e6-400b-9497-246ae3d52b33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:12.567307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94868061-ef79-4af3-910b-a71f19076313 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:12.568006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94868061-ef79-4af3-910b-a71f19076313 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:12.577726 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc12e09-66e6-400b-9497-246ae3d52b33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:12.579173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc12e09-66e6-400b-9497-246ae3d52b33 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:12.593887 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94868061-ef79-4af3-910b-a71f19076313 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:12.597591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-8bc12e09-66e6-400b-9497-246ae3d52b33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:12.597591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:12.597591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:12.597591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:12.597591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:12.597591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:12.597591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:12.597591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:12.597591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:12.597591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:12.597591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:12.597591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:12.597591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:12.597591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:12.597591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:12.599622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:12.601740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:12.601740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:12.601740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:12.601740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:12.601740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 283/548] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-94868061-ef79-4af3-910b-a71f19076313 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:12.602409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:12.609010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:12.611515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:12.611515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:12.611515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:12.611515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:12.611515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:12.611515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:12.611515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:12.611515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:12.611515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:12.611515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:12.611515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:12.611515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:12.613181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc12e09-66e6-400b-9497-246ae3d52b33 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:12.613334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 266/549] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:12.631040 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94868061-ef79-4af3-910b-a71f19076313 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:12.718067 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:12.760612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc12e09-66e6-400b-9497-246ae3d52b33 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:12.777133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94868061-ef79-4af3-910b-a71f19076313 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:12.819704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5325b-a10b-41a4-b794-1cc66c559db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:12.820566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5325b-a10b-41a4-b794-1cc66c559db8 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:12.820854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5325b-a10b-41a4-b794-1cc66c559db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:12.821165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5325b-a10b-41a4-b794-1cc66c559db8 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:12.821983 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5325b-a10b-41a4-b794-1cc66c559db8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:12.822294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5325b-a10b-41a4-b794-1cc66c559db8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:12.824119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5325b-a10b-41a4-b794-1cc66c559db8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:12.834058 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 267/550] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:12.840643 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e1b41-76eb-4914-8556-bd7ee0db9ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:12.841368 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e1b41-76eb-4914-8556-bd7ee0db9ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:12.841653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e1b41-76eb-4914-8556-bd7ee0db9ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:12.841932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3e1b41-76eb-4914-8556-bd7ee0db9ea4 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:12.842805 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e1b41-76eb-4914-8556-bd7ee0db9ea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:12.844066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e1b41-76eb-4914-8556-bd7ee0db9ea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:12.845481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3e1b41-76eb-4914-8556-bd7ee0db9ea4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:12.845953 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5325b-a10b-41a4-b794-1cc66c559db8 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:12.861656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 284/551] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:12.867021 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e1b41-76eb-4914-8556-bd7ee0db9ea4 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:12.893375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:13.004682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3e1b41-76eb-4914-8556-bd7ee0db9ea4 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:13.013741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5325b-a10b-41a4-b794-1cc66c559db8 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:13.046733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:13.051612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a875f8e4-9054-4e9f-b2b1-6d552e8341d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:13.052583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875f8e4-9054-4e9f-b2b1-6d552e8341d8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:13.053037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875f8e4-9054-4e9f-b2b1-6d552e8341d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:13.053482 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a875f8e4-9054-4e9f-b2b1-6d552e8341d8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:13.061851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875f8e4-9054-4e9f-b2b1-6d552e8341d8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:13.062271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddaf1de-de6d-4a9b-a96a-fd83186efe50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:13.063162 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaf1de-de6d-4a9b-a96a-fd83186efe50 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:13.063544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaf1de-de6d-4a9b-a96a-fd83186efe50 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:13.063917 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddaf1de-de6d-4a9b-a96a-fd83186efe50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:13.064246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a875f8e4-9054-4e9f-b2b1-6d552e8341d8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:13.077149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddaf1de-de6d-4a9b-a96a-fd83186efe50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:13.078699 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddaf1de-de6d-4a9b-a96a-fd83186efe50 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-a875f8e4-9054-4e9f-b2b1-6d552e8341d8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:13.080042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:13.081151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:13.082185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:13.082185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:13.082185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:13.082185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:13.082185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:13.082185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:13.082185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:13.082185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:13.082185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:01:13.082185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:13.084653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 285/552] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:01:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:13.086253 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cae28-14d0-46f4-adb5-ee78cc04a5c8 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:13.086718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cae28-14d0-46f4-adb5-ee78cc04a5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:13.087165 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96cae28-14d0-46f4-adb5-ee78cc04a5c8 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:13.093010 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 286/553] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:13.094056 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cae28-14d0-46f4-adb5-ee78cc04a5c8 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:13.106219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-5ddaf1de-de6d-4a9b-a96a-fd83186efe50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:13.106219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:13.106219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:13.106219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:13.106219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:13.106219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:13.106219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:13.106219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:13.106219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:13.106219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:13.106219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:13.106219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:13.106219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:13.106219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:13.106219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:13.107093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:13.107968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:13.107968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:13.107968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:13.107968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:13.107968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 268/554] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:13.108888 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb193c-2405-47a2-9d44-4ae91ff9339d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:13.113301 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb193c-2405-47a2-9d44-4ae91ff9339d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:13.114468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb193c-2405-47a2-9d44-4ae91ff9339d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:13.133954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f8cb193c-2405-47a2-9d44-4ae91ff9339d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:13.136054 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f8cb193c-2405-47a2-9d44-4ae91ff9339d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:13.187703 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:13.213028 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 269/555] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:13.213028 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb193c-2405-47a2-9d44-4ae91ff9339d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:13.217613 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96cae28-14d0-46f4-adb5-ee78cc04a5c8 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:13.267956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d99c73-628d-43be-bad6-37fde568e165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:13.268564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d99c73-628d-43be-bad6-37fde568e165 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:13.268842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d99c73-628d-43be-bad6-37fde568e165 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:13.269147 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d99c73-628d-43be-bad6-37fde568e165 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:13.269990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d99c73-628d-43be-bad6-37fde568e165 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:13.270297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d99c73-628d-43be-bad6-37fde568e165 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:13.271722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d99c73-628d-43be-bad6-37fde568e165 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:13.290182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 287/556] 10.222.0.61 () {56 vars in 1135 bytes} [Thu Jul 14 10:01:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:13.291053 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d99c73-628d-43be-bad6-37fde568e165 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:13.324862 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb193c-2405-47a2-9d44-4ae91ff9339d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:13.332806 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:13.363033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee23c78-573d-4710-a99c-f6b4e4e2188d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:13.363729 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee23c78-573d-4710-a99c-f6b4e4e2188d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:13.364012 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee23c78-573d-4710-a99c-f6b4e4e2188d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:13.364290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee23c78-573d-4710-a99c-f6b4e4e2188d None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:13.365037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee23c78-573d-4710-a99c-f6b4e4e2188d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:13.365350 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee23c78-573d-4710-a99c-f6b4e4e2188d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:13.366828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee23c78-573d-4710-a99c-f6b4e4e2188d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:13.379925 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 270/557] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:13.380779 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee23c78-573d-4710-a99c-f6b4e4e2188d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:13.399116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d99c73-628d-43be-bad6-37fde568e165 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:13.434652 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ea659-cccd-4574-beb7-1d9cda086269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:13.435717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ea659-cccd-4574-beb7-1d9cda086269 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:13.435999 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ea659-cccd-4574-beb7-1d9cda086269 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:13.436273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108ea659-cccd-4574-beb7-1d9cda086269 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:13.443682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:13.449705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108ea659-cccd-4574-beb7-1d9cda086269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-108ea659-cccd-4574-beb7-1d9cda086269 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:13.457878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:13.459548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:13.460562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:13.460562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:13.460562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:13.460562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:13.460562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:13.460562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:13.460562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:13.460562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:13.460562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:13.460562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:13.460562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:13.460562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:13.468121 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 288/558] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:13.469264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ea659-cccd-4574-beb7-1d9cda086269 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:13.502477 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee23c78-573d-4710-a99c-f6b4e4e2188d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:13.540222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00355215-6503-445f-b19d-6d68c9158e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:13.540744 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00355215-6503-445f-b19d-6d68c9158e12 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:13.541054 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00355215-6503-445f-b19d-6d68c9158e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:13.541347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00355215-6503-445f-b19d-6d68c9158e12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:13.577961 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:13.578586 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108ea659-cccd-4574-beb7-1d9cda086269 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:13.613603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b8f5b-8952-4117-b9c6-631b4b163a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:13.614639 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b8f5b-8952-4117-b9c6-631b4b163a1a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:13.615091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b8f5b-8952-4117-b9c6-631b4b163a1a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:13.615507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b8f5b-8952-4117-b9c6-631b4b163a1a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:13.615913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00355215-6503-445f-b19d-6d68c9158e12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:13.617827 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00355215-6503-445f-b19d-6d68c9158e12 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:13.624500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b8f5b-8952-4117-b9c6-631b4b163a1a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:13.626206 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b8f5b-8952-4117-b9c6-631b4b163a1a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-ef2b8f5b-8952-4117-b9c6-631b4b163a1a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:13.641237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:13.642487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:13.643552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:13.643552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:13.643552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:13.643552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:13.643552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:13.643552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:13.643552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:13.643552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:13.643552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:01:13.643552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:13.645759 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 289/559] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Jul 14 10:01:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:13.646711 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b8f5b-8952-4117-b9c6-631b4b163a1a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:13.723673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:13.761393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 271/560] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:01:13.762412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00355215-6503-445f-b19d-6d68c9158e12 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:13.771658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b8f5b-8952-4117-b9c6-631b4b163a1a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:13.806916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335f47b-78e0-443a-ab5d-5b833909bc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:13.807834 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335f47b-78e0-443a-ab5d-5b833909bc4f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:13.808115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335f47b-78e0-443a-ab5d-5b833909bc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:13.808387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5335f47b-78e0-443a-ab5d-5b833909bc4f None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:13.809164 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335f47b-78e0-443a-ab5d-5b833909bc4f None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:13.809475 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335f47b-78e0-443a-ab5d-5b833909bc4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:13.811092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5335f47b-78e0-443a-ab5d-5b833909bc4f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:13.821288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 290/561] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:13.821288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335f47b-78e0-443a-ab5d-5b833909bc4f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:13.837408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:13.875496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00355215-6503-445f-b19d-6d68c9158e12 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:13.912019 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca740de4-5c65-4d25-9fdb-53342c20eeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:13.912686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca740de4-5c65-4d25-9fdb-53342c20eeb0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:13.912991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca740de4-5c65-4d25-9fdb-53342c20eeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:13.913266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca740de4-5c65-4d25-9fdb-53342c20eeb0 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:13.914160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca740de4-5c65-4d25-9fdb-53342c20eeb0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:13.914470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca740de4-5c65-4d25-9fdb-53342c20eeb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:13.916522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca740de4-5c65-4d25-9fdb-53342c20eeb0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:13.925345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5335f47b-78e0-443a-ab5d-5b833909bc4f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:13.935010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 272/562] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Jul 14 10:01:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:13.936549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86a8ea-819c-45b7-b781-e4932a946ff9 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:13.936840 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86a8ea-819c-45b7-b781-e4932a946ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:13.937143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f86a8ea-819c-45b7-b781-e4932a946ff9 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:13.938493 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 273/563] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:13.941977 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86a8ea-819c-45b7-b781-e4932a946ff9 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:13.951051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:13.966566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda484d-0541-4066-98b7-96506bdfa8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:13.967471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda484d-0541-4066-98b7-96506bdfa8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:13.967817 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda484d-0541-4066-98b7-96506bdfa8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:13.968162 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda484d-0541-4066-98b7-96506bdfa8d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:13.979815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda484d-0541-4066-98b7-96506bdfa8d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:13.981334 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dda484d-0541-4066-98b7-96506bdfa8d0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:13.999197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-3dda484d-0541-4066-98b7-96506bdfa8d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:13.999197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:13.999197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:13.999197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:13.999197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:13.999197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:13.999197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:13.999197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:13.999197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:13.999197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:13.999197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:13.999197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:13.999197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:13.999197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:13.999197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:14.000096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:14.001040 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:14.001040 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:14.001040 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:14.001040 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:14.005309 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 291/564] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:14.005938 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda484d-0541-4066-98b7-96506bdfa8d0 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:14.054520 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f86a8ea-819c-45b7-b781-e4932a946ff9 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:14.082405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:14.093108 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e259ceac-e1fe-45fe-bc64-df2cad184279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:14.093923 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259ceac-e1fe-45fe-bc64-df2cad184279 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:14.094199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259ceac-e1fe-45fe-bc64-df2cad184279 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:14.094469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e259ceac-e1fe-45fe-bc64-df2cad184279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:14.106773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259ceac-e1fe-45fe-bc64-df2cad184279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:14.108237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e259ceac-e1fe-45fe-bc64-df2cad184279 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:14.118721 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda484d-0541-4066-98b7-96506bdfa8d0 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:14.126551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e259ceac-e1fe-45fe-bc64-df2cad184279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:14.126551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:14.126551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:14.126551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:14.126551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:14.126551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:14.126551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:14.126551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:14.126551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:14.126551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:14.126551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:14.126551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:14.126551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:14.126551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:14.126551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:14.127453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:14.128374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:14.128374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:14.128374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:14.128374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:14.128374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 274/565] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:14.129475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9f9e2-e143-4446-ae57-dfd77a1ada19 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:14.129652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9f9e2-e143-4446-ae57-dfd77a1ada19 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:14.129881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9f9e2-e143-4446-ae57-dfd77a1ada19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:14.146209 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b1f9f9e2-e143-4446-ae57-dfd77a1ada19 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:14.147393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b1f9f9e2-e143-4446-ae57-dfd77a1ada19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:14.158958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3429f-e38a-427a-b0d2-f6b18c5507ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:14.159981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3429f-e38a-427a-b0d2-f6b18c5507ac None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:14.160472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3429f-e38a-427a-b0d2-f6b18c5507ac None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:14.161067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3429f-e38a-427a-b0d2-f6b18c5507ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:14.173749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3429f-e38a-427a-b0d2-f6b18c5507ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:14.175254 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d3429f-e38a-427a-b0d2-f6b18c5507ac None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:14.194246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-b0d3429f-e38a-427a-b0d2-f6b18c5507ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:14.194246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:14.194246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:14.194246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:14.194246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:14.194246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:14.194246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:14.194246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:14.194246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:14.194246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:14.194246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:14.194246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:14.194246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:14.194246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:14.194246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:14.195354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:14.196360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:14.196360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:14.196360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:14.196360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:14.196360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 292/566] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:14.197481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3429f-e38a-427a-b0d2-f6b18c5507ac None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:14.222594 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 275/567] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:14.223642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9f9e2-e143-4446-ae57-dfd77a1ada19 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:14.229234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:14.316225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d3429f-e38a-427a-b0d2-f6b18c5507ac None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:14.341465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9f9e2-e143-4446-ae57-dfd77a1ada19 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:14.349287 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:14.349958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a62241-666f-4ed7-9bbb-0c13d9d259c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:14.350776 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a62241-666f-4ed7-9bbb-0c13d9d259c7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:14.350926 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a62241-666f-4ed7-9bbb-0c13d9d259c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:14.351197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a62241-666f-4ed7-9bbb-0c13d9d259c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:14.351983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a62241-666f-4ed7-9bbb-0c13d9d259c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:14.352289 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a62241-666f-4ed7-9bbb-0c13d9d259c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:14.353676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a62241-666f-4ed7-9bbb-0c13d9d259c7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:14.370892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 293/568] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:14.370892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a62241-666f-4ed7-9bbb-0c13d9d259c7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:14.387714 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9703c26-589c-4f96-b86f-7af26f69f322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:14.388392 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9703c26-589c-4f96-b86f-7af26f69f322 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:14.388675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9703c26-589c-4f96-b86f-7af26f69f322 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:14.389260 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9703c26-589c-4f96-b86f-7af26f69f322 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:14.389701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9703c26-589c-4f96-b86f-7af26f69f322 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:14.390010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9703c26-589c-4f96-b86f-7af26f69f322 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:14.391374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9703c26-589c-4f96-b86f-7af26f69f322 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:14.403818 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 276/569] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:14.408760 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9703c26-589c-4f96-b86f-7af26f69f322 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:14.454133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:14.470110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a62241-666f-4ed7-9bbb-0c13d9d259c7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:14.503705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f6224-d581-417d-9335-9a41de411ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:14.504369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f6224-d581-417d-9335-9a41de411ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:14.504651 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f6224-d581-417d-9335-9a41de411ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:14.504949 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f6224-d581-417d-9335-9a41de411ea1 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:14.505674 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f6224-d581-417d-9335-9a41de411ea1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:14.505980 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f6224-d581-417d-9335-9a41de411ea1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:14.507352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f6224-d581-417d-9335-9a41de411ea1 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:14.511620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9703c26-589c-4f96-b86f-7af26f69f322 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:14.519371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 294/570] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:14.520161 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f6224-d581-417d-9335-9a41de411ea1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:14.546123 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2676e-ce98-486f-bcb6-e55b2ec67352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:14.546919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2676e-ce98-486f-bcb6-e55b2ec67352 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:14.547201 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2676e-ce98-486f-bcb6-e55b2ec67352 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:14.547478 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb2676e-ce98-486f-bcb6-e55b2ec67352 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:14.558222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb2676e-ce98-486f-bcb6-e55b2ec67352 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-bcb2676e-ce98-486f-bcb6-e55b2ec67352 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:14.566158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:14.567097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:14.567945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:14.567945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:14.567945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:14.567945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:14.567945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:14.567945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:14.567945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:14.567945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:14.567945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:14.567945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:14.567945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:14.567945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:14.567945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 277/571] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:14.569342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2676e-ce98-486f-bcb6-e55b2ec67352 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:14.595853 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:14.622631 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f6224-d581-417d-9335-9a41de411ea1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:14.656791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c1531-9002-4a4d-8041-53d831c861b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:14.657611 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c1531-9002-4a4d-8041-53d831c861b6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:14.657894 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c1531-9002-4a4d-8041-53d831c861b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:14.658171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c1531-9002-4a4d-8041-53d831c861b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:14.664647 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c1531-9002-4a4d-8041-53d831c861b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:14.666201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c1531-9002-4a4d-8041-53d831c861b6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:14.668521 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb2676e-ce98-486f-bcb6-e55b2ec67352 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:14.679696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-e95c1531-9002-4a4d-8041-53d831c861b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:14.679696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:14.679696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:14.679696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:14.679696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:14.679696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:14.679696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:14.679696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:14.679696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:14.679696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:14.679696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:14.679696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:14.679696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:14.679696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:14.679696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:14.680715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:14.681705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:14.681705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:14.681705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:14.681705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:14.681705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:14.681705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:14.681705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:14.681705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:14.681705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 295/572] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:14.682633 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706aba1-34d9-4349-9bea-19572359645f None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:14.682871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706aba1-34d9-4349-9bea-19572359645f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:14.683148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2706aba1-34d9-4349-9bea-19572359645f None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:14.684397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 296/573] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:14.685430 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706aba1-34d9-4349-9bea-19572359645f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:14.689745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:14.708201 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcf93c-fbc6-4f0c-9995-7a1cef18502a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:14.709021 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcf93c-fbc6-4f0c-9995-7a1cef18502a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:14.709307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcf93c-fbc6-4f0c-9995-7a1cef18502a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:14.709584 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebcf93c-fbc6-4f0c-9995-7a1cef18502a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:14.717685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebcf93c-fbc6-4f0c-9995-7a1cef18502a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:14.719139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebcf93c-fbc6-4f0c-9995-7a1cef18502a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-5ebcf93c-fbc6-4f0c-9995-7a1cef18502a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:14.733574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:14.734515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:14.735464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:14.735464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:14.735464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:14.735464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:14.735464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:14.735464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:14.735464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:14.735464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:14.735464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:01:14.735464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:14.735464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 278/574] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:01:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:14.736042 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcf93c-fbc6-4f0c-9995-7a1cef18502a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:14.790669 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2706aba1-34d9-4349-9bea-19572359645f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:14.819694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:14.824527 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be542a56-df52-4929-ab88-77d8c4641a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:14.825531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be542a56-df52-4929-ab88-77d8c4641a00 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:14.825816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be542a56-df52-4929-ab88-77d8c4641a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:14.826089 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be542a56-df52-4929-ab88-77d8c4641a00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:14.832655 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be542a56-df52-4929-ab88-77d8c4641a00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:14.834113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be542a56-df52-4929-ab88-77d8c4641a00 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:14.835823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebcf93c-fbc6-4f0c-9995-7a1cef18502a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:14.848047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-be542a56-df52-4929-ab88-77d8c4641a00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:14.848047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:14.848047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:14.848047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:14.848047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:14.848047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:14.848047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:14.848047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:14.848047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:14.848047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:14.848047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:14.848047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:14.848047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:14.848047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:14.848047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:14.849119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:14.850012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:14.850012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:14.850012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:14.850012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:14.850012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:14.850012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:14.850012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:14.850012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:14.850012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 297/575] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:14.850510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be542a56-df52-4929-ab88-77d8c4641a00 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:14.874694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd4188f-a0a9-4570-af33-5b79cb20e27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:14.875358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4188f-a0a9-4570-af33-5b79cb20e27c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:14.875641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4188f-a0a9-4570-af33-5b79cb20e27c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:14.875912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd4188f-a0a9-4570-af33-5b79cb20e27c None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:14.876658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4188f-a0a9-4570-af33-5b79cb20e27c None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:14.876993 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4188f-a0a9-4570-af33-5b79cb20e27c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:14.878358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd4188f-a0a9-4570-af33-5b79cb20e27c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:14.886296 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 279/576] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:14.887542 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b235d-16b0-480a-95ab-474d3e3d5557 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:14.887834 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b235d-16b0-480a-95ab-474d3e3d5557 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:14.888115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b235d-16b0-480a-95ab-474d3e3d5557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:14.905279 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a20b235d-16b0-480a-95ab-474d3e3d5557 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:14.906451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a20b235d-16b0-480a-95ab-474d3e3d5557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:14.909648 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:14.946697 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be542a56-df52-4929-ab88-77d8c4641a00 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:14.972005 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 280/577] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:14.972830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b235d-16b0-480a-95ab-474d3e3d5557 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:14.994225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bedb1-203f-4038-9715-d35bd543cb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:14.995290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bedb1-203f-4038-9715-d35bd543cb48 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:14.995568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bedb1-203f-4038-9715-d35bd543cb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:14.995839 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287bedb1-203f-4038-9715-d35bd543cb48 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:14.996651 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bedb1-203f-4038-9715-d35bd543cb48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:14.996983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bedb1-203f-4038-9715-d35bd543cb48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:14.998401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287bedb1-203f-4038-9715-d35bd543cb48 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:15.010808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 298/578] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:15.012228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bedb1-203f-4038-9715-d35bd543cb48 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:15.051094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:15.073319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b235d-16b0-480a-95ab-474d3e3d5557 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:15.107200 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6bd23a-8380-4c4a-bc78-2805b6ef9965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:15.107791 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bd23a-8380-4c4a-bc78-2805b6ef9965 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:15.108067 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bd23a-8380-4c4a-bc78-2805b6ef9965 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:15.108336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6bd23a-8380-4c4a-bc78-2805b6ef9965 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:15.109211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6bd23a-8380-4c4a-bc78-2805b6ef9965 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:15.109518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6bd23a-8380-4c4a-bc78-2805b6ef9965 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:15.110935 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6bd23a-8380-4c4a-bc78-2805b6ef9965 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:15.113173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287bedb1-203f-4038-9715-d35bd543cb48 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:15.127659 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 281/579] 10.222.0.61 () {56 vars in 1137 bytes} [Thu Jul 14 10:01:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:15.128451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6bd23a-8380-4c4a-bc78-2805b6ef9965 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:15.136573 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:15.148492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787f527-e5cc-4d8f-8134-90dddc3b7c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:15.149198 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787f527-e5cc-4d8f-8134-90dddc3b7c7b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:15.150029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787f527-e5cc-4d8f-8134-90dddc3b7c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:15.150306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787f527-e5cc-4d8f-8134-90dddc3b7c7b None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:15.151097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787f527-e5cc-4d8f-8134-90dddc3b7c7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:15.151401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787f527-e5cc-4d8f-8134-90dddc3b7c7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:15.152833 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787f527-e5cc-4d8f-8134-90dddc3b7c7b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:15.164040 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 299/580] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:15.164844 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787f527-e5cc-4d8f-8134-90dddc3b7c7b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:15.230387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6bd23a-8380-4c4a-bc78-2805b6ef9965 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:15.263921 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787f527-e5cc-4d8f-8134-90dddc3b7c7b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:15.265883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a782a5e-8af3-4a4b-8ea2-f201cb4ddb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:15.266677 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a782a5e-8af3-4a4b-8ea2-f201cb4ddb37 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:15.266955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a782a5e-8af3-4a4b-8ea2-f201cb4ddb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:15.267225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a782a5e-8af3-4a4b-8ea2-f201cb4ddb37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:15.269895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:15.278520 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a782a5e-8af3-4a4b-8ea2-f201cb4ddb37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:15.279956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a782a5e-8af3-4a4b-8ea2-f201cb4ddb37 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:15.297720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-5a782a5e-8af3-4a4b-8ea2-f201cb4ddb37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:15.297720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:15.297720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:15.297720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:15.297720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:15.297720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:15.297720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:15.297720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:15.297720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:15.297720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:15.297720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:15.297720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:15.297720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:15.297720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:15.297720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:15.298578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:15.299542 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:15.299542 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:15.299542 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:15.299542 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:15.299761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27696d04-237b-4daf-aa74-77c0517a4300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:15.299761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27696d04-237b-4daf-aa74-77c0517a4300 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:15.299916 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 282/581] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:15.299984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27696d04-237b-4daf-aa74-77c0517a4300 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:15.299984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27696d04-237b-4daf-aa74-77c0517a4300 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:15.300103 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a782a5e-8af3-4a4b-8ea2-f201cb4ddb37 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:15.311235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27696d04-237b-4daf-aa74-77c0517a4300 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-27696d04-237b-4daf-aa74-77c0517a4300 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:15.318966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:15.319969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:15.320820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:15.320820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:15.320820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:15.320820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:15.320820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:15.320820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:15.320820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:15.320820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:15.320820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:15.320820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:15.320820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:15.320820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:15.320820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 300/582] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:15.322070 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27696d04-237b-4daf-aa74-77c0517a4300 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:15.365760 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:15.401523 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a782a5e-8af3-4a4b-8ea2-f201cb4ddb37 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:15.419789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27696d04-237b-4daf-aa74-77c0517a4300 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:15.434092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500baf64-ffd6-4f61-a608-98ff81cbb128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:15.434993 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500baf64-ffd6-4f61-a608-98ff81cbb128 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:15.435281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500baf64-ffd6-4f61-a608-98ff81cbb128 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:15.435552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500baf64-ffd6-4f61-a608-98ff81cbb128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:15.436374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500baf64-ffd6-4f61-a608-98ff81cbb128 None admin] RBAC: Adding query filter params (role.id=ba4088a289ea4b9a9aa9d9cd7a27e731, scope.project.id=57662256880f43018039715a755c087c, user.id=d5fa502e8b014c719fcece0a130fdd7e) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:15.436680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500baf64-ffd6-4f61-a608-98ff81cbb128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:15.438141 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500baf64-ffd6-4f61-a608-98ff81cbb128 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:15.443678 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 283/583] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:15 2022] GET /identity/v3/role_assignments?user.id=d5fa502e8b014c719fcece0a130fdd7e&scope.project.id=57662256880f43018039715a755c087c&role.id=ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:15.444374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500baf64-ffd6-4f61-a608-98ff81cbb128 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:15.452690 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1abc5c-bf78-4136-850c-f0f7bc790303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:15.453530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1abc5c-bf78-4136-850c-f0f7bc790303 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:15.453809 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1abc5c-bf78-4136-850c-f0f7bc790303 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:15.454079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1abc5c-bf78-4136-850c-f0f7bc790303 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:15.467343 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1abc5c-bf78-4136-850c-f0f7bc790303 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:15.469170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1abc5c-bf78-4136-850c-f0f7bc790303 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:15.489435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-3e1abc5c-bf78-4136-850c-f0f7bc790303 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:15.489435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:15.489435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:15.489435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:15.489435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:15.489435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:15.489435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:15.489435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:15.489435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:15.489435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:15.489435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:15.489435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:15.489435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:15.489435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:15.489435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:15.490429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:15.491321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:15.491321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:15.491321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:15.491321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:15.491321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 301/584] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:15.491610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1abc5c-bf78-4136-850c-f0f7bc790303 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:15.525037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:15.580734 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500baf64-ffd6-4f61-a608-98ff81cbb128 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:15.599437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:15.605998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1abc5c-bf78-4136-850c-f0f7bc790303 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:15.620776 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e748d1-0355-4c8e-91e5-48827add3bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:15.621627 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e748d1-0355-4c8e-91e5-48827add3bec None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:15.621914 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e748d1-0355-4c8e-91e5-48827add3bec None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:15.622182 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e748d1-0355-4c8e-91e5-48827add3bec None admin] PATH_INFO: `/v3/projects/57662256880f43018039715a755c087c/users/e9f4efa3e2f343c2ac858a277601a7df/roles/ba4088a289ea4b9a9aa9d9cd7a27e731` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:15.637616 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e748d1-0355-4c8e-91e5-48827add3bec None admin] RBAC: Authorizing `identity:create_grant(project_id=57662256880f43018039715a755c087c, user_id=e9f4efa3e2f343c2ac858a277601a7df, role_id=ba4088a289ea4b9a9aa9d9cd7a27e731)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:15.639046 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e748d1-0355-4c8e-91e5-48827add3bec None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:15.644786 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9b5d7-ba4e-40f6-a325-20fd34bee2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:15.645717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9b5d7-ba4e-40f6-a325-20fd34bee2ac None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:15.646041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9b5d7-ba4e-40f6-a325-20fd34bee2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:15.646474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9b5d7-ba4e-40f6-a325-20fd34bee2ac None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:15.647209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9b5d7-ba4e-40f6-a325-20fd34bee2ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:15.647517 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9b5d7-ba4e-40f6-a325-20fd34bee2ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:15.648942 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f9b5d7-ba4e-40f6-a325-20fd34bee2ac None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:15.664952 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 284/585] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:01:15 2022] PUT /identity/v3/projects/57662256880f43018039715a755c087c/users/e9f4efa3e2f343c2ac858a277601a7df/roles/ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:01:15.664952 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e748d1-0355-4c8e-91e5-48827add3bec None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:15.672012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 302/586] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:15.673101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9b5d7-ba4e-40f6-a325-20fd34bee2ac None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:15.800074 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9b5d7-ba4e-40f6-a325-20fd34bee2ac None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:15.816217 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e748d1-0355-4c8e-91e5-48827add3bec None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:15.818651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:15.848569 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30123c-0e53-4741-bdf6-1c3e953111d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:15.850654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30123c-0e53-4741-bdf6-1c3e953111d0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:15.850654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30123c-0e53-4741-bdf6-1c3e953111d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:15.850654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f30123c-0e53-4741-bdf6-1c3e953111d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:15.850654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30123c-0e53-4741-bdf6-1c3e953111d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:15.850976 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30123c-0e53-4741-bdf6-1c3e953111d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:15.852246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f30123c-0e53-4741-bdf6-1c3e953111d0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:15.864528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 303/587] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:15.868370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b334d40-2946-4c59-a680-42d29e8f43eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:15.868535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30123c-0e53-4741-bdf6-1c3e953111d0 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:15.868632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b334d40-2946-4c59-a680-42d29e8f43eb None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:15.868632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b334d40-2946-4c59-a680-42d29e8f43eb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:15.868632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b334d40-2946-4c59-a680-42d29e8f43eb None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:15.868632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b334d40-2946-4c59-a680-42d29e8f43eb None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:15.868632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b334d40-2946-4c59-a680-42d29e8f43eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:15.869058 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b334d40-2946-4c59-a680-42d29e8f43eb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:15.878638 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 285/588] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:15.879258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b334d40-2946-4c59-a680-42d29e8f43eb None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:15.895676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:15.990791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f30123c-0e53-4741-bdf6-1c3e953111d0 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:16.023572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b334d40-2946-4c59-a680-42d29e8f43eb None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:16.025490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195ef19-8e33-4957-9654-3617591f40f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:16.026360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195ef19-8e33-4957-9654-3617591f40f1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:16.026715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195ef19-8e33-4957-9654-3617591f40f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:16.027055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195ef19-8e33-4957-9654-3617591f40f1 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:16.038424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195ef19-8e33-4957-9654-3617591f40f1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:16.041415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195ef19-8e33-4957-9654-3617591f40f1 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:16.062713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5652ca-f3ac-4187-a171-2a56e0c9aa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-5195ef19-8e33-4957-9654-3617591f40f1 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:16.063234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:16.064215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:16.065202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:16.065202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:16.065202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:16.065202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:16.065202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:16.065202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:16.065202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:16.065202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:16.065202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:01:16.065202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:16.065740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5652ca-f3ac-4187-a171-2a56e0c9aa20 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:16.065740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5652ca-f3ac-4187-a171-2a56e0c9aa20 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:16.065740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5652ca-f3ac-4187-a171-2a56e0c9aa20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:16.066613 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 304/589] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:01:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:16.067411 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195ef19-8e33-4957-9654-3617591f40f1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:16.075535 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5652ca-f3ac-4187-a171-2a56e0c9aa20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:16.078268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5652ca-f3ac-4187-a171-2a56e0c9aa20 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:16.100646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-cf5652ca-f3ac-4187-a171-2a56e0c9aa20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:16.100646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:16.100646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:16.100646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:16.100646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:16.100646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:16.100646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:16.100646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:16.100646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:16.100646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:16.100646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:16.100646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:16.100646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:16.100646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:16.100646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:16.101696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:16.103172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:16.103172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:16.103172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:16.103172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:16.103172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:16.103172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:16.103172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:16.103172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:16.110094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 286/590] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:16.110094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5652ca-f3ac-4187-a171-2a56e0c9aa20 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:16.138702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:16.171392 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:16.208539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195ef19-8e33-4957-9654-3617591f40f1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:16.230839 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5652ca-f3ac-4187-a171-2a56e0c9aa20 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:16.270609 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903df02-eb5f-4c88-b927-b2aa65aa25d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:16.271996 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903df02-eb5f-4c88-b927-b2aa65aa25d3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:16.272836 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903df02-eb5f-4c88-b927-b2aa65aa25d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:16.273622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903df02-eb5f-4c88-b927-b2aa65aa25d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:16.280821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e2cf8-20bf-4138-a341-f80db5a3e283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:16.281520 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e2cf8-20bf-4138-a341-f80db5a3e283 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:16.281801 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e2cf8-20bf-4138-a341-f80db5a3e283 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:16.282074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3e2cf8-20bf-4138-a341-f80db5a3e283 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:16.283010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e2cf8-20bf-4138-a341-f80db5a3e283 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:16.283222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e2cf8-20bf-4138-a341-f80db5a3e283 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:16.284652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3e2cf8-20bf-4138-a341-f80db5a3e283 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:16.288761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903df02-eb5f-4c88-b927-b2aa65aa25d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:16.290985 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903df02-eb5f-4c88-b927-b2aa65aa25d3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:16.307924 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 287/591] 10.222.0.61 () {56 vars in 1137 bytes} [Thu Jul 14 10:01:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:16.307924 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e2cf8-20bf-4138-a341-f80db5a3e283 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:16.318998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-b903df02-eb5f-4c88-b927-b2aa65aa25d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:16.318998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:16.318998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:16.318998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:16.318998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:16.318998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:16.318998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:16.318998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:16.318998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:16.318998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:16.318998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:16.318998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:16.318998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:16.318998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:16.318998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:16.320047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:16.321233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:16.321233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:16.321233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:16.321233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:16.327369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 305/592] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:16.327369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903df02-eb5f-4c88-b927-b2aa65aa25d3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:16.409057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:16.431279 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3e2cf8-20bf-4138-a341-f80db5a3e283 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:16.465760 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903df02-eb5f-4c88-b927-b2aa65aa25d3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:16.467646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a7bd1-9337-4b1d-92ba-0d31321b0675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:16.468262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a7bd1-9337-4b1d-92ba-0d31321b0675 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:16.468549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a7bd1-9337-4b1d-92ba-0d31321b0675 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:16.468823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037a7bd1-9337-4b1d-92ba-0d31321b0675 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:16.470048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a7bd1-9337-4b1d-92ba-0d31321b0675 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:16.470048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a7bd1-9337-4b1d-92ba-0d31321b0675 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:16.471383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037a7bd1-9337-4b1d-92ba-0d31321b0675 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:16.489188 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 288/593] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:16.490066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a7bd1-9337-4b1d-92ba-0d31321b0675 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:16.499982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:16.511709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7e0ed-c283-4afd-8278-1b1891a20969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:16.512806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7e0ed-c283-4afd-8278-1b1891a20969 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:16.513334 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7e0ed-c283-4afd-8278-1b1891a20969 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:16.513803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7e0ed-c283-4afd-8278-1b1891a20969 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:16.530554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f7e0ed-c283-4afd-8278-1b1891a20969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:16.532397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f7e0ed-c283-4afd-8278-1b1891a20969 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:16.557880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-98f7e0ed-c283-4afd-8278-1b1891a20969 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:16.557880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:16.557880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:16.557880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:16.557880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:16.557880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:16.557880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:16.557880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:16.557880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:16.557880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:16.557880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:16.557880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:16.557880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:16.557880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:16.557880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:16.558842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:16.559997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:16.559997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:16.559997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:16.559997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:16.561856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 306/594] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:16.561856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7e0ed-c283-4afd-8278-1b1891a20969 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:16.610263 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037a7bd1-9337-4b1d-92ba-0d31321b0675 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:16.649183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ea134-656a-4fe4-a37c-f63d8fd57386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:16.649537 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ea134-656a-4fe4-a37c-f63d8fd57386 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:16.649821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ea134-656a-4fe4-a37c-f63d8fd57386 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:16.650096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ea134-656a-4fe4-a37c-f63d8fd57386 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:16.650870 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ea134-656a-4fe4-a37c-f63d8fd57386 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:16.651183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ea134-656a-4fe4-a37c-f63d8fd57386 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:16.652608 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ea134-656a-4fe4-a37c-f63d8fd57386 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:16.665607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 289/595] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:16.666682 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ea134-656a-4fe4-a37c-f63d8fd57386 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:16.672070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:16.687047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f7e0ed-c283-4afd-8278-1b1891a20969 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:16.722081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5444d-8319-427b-bbb0-399d4cdc11a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:16.723300 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5444d-8319-427b-bbb0-399d4cdc11a5 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:16.723800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5444d-8319-427b-bbb0-399d4cdc11a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:16.724272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5444d-8319-427b-bbb0-399d4cdc11a5 None admin] PATH_INFO: `/v3/projects/57662256880f43018039715a755c087c/users/52e3fbd0400a49a69932ce426218f4a6/roles/ba4088a289ea4b9a9aa9d9cd7a27e731` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:16.739659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5444d-8319-427b-bbb0-399d4cdc11a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=57662256880f43018039715a755c087c, user_id=52e3fbd0400a49a69932ce426218f4a6, role_id=ba4088a289ea4b9a9aa9d9cd7a27e731)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:16.741382 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5444d-8319-427b-bbb0-399d4cdc11a5 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:16.759391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 307/596] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:01:16 2022] PUT /identity/v3/projects/57662256880f43018039715a755c087c/users/52e3fbd0400a49a69932ce426218f4a6/roles/ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:01:16.760743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5444d-8319-427b-bbb0-399d4cdc11a5 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:16.814976 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ea134-656a-4fe4-a37c-f63d8fd57386 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:16.828094 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:16.908852 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5444d-8319-427b-bbb0-399d4cdc11a5 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:16.925775 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa80b15-bdcb-4614-bf58-0795417d433e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:16.926378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa80b15-bdcb-4614-bf58-0795417d433e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:16.926659 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa80b15-bdcb-4614-bf58-0795417d433e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:16.926933 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa80b15-bdcb-4614-bf58-0795417d433e None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:16.927675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa80b15-bdcb-4614-bf58-0795417d433e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:16.928052 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa80b15-bdcb-4614-bf58-0795417d433e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:16.929404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa80b15-bdcb-4614-bf58-0795417d433e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:16.952966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 290/597] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:16.958425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa80b15-bdcb-4614-bf58-0795417d433e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:16.960212 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cbea8-821f-444d-ab8d-9479b93f7204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:16.961132 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cbea8-821f-444d-ab8d-9479b93f7204 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:16.961606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cbea8-821f-444d-ab8d-9479b93f7204 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:16.961953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1cbea8-821f-444d-ab8d-9479b93f7204 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:16.971731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1cbea8-821f-444d-ab8d-9479b93f7204 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:16.973303 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1cbea8-821f-444d-ab8d-9479b93f7204 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-9d1cbea8-821f-444d-ab8d-9479b93f7204 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:16.991709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:16.994024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:16.995415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:16.995415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:16.995415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:16.995415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:16.995415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:16.995415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:16.995415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:16.995415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:16.995415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:01:16.995415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:16.995415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 308/598] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:17.000788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cbea8-821f-444d-ab8d-9479b93f7204 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:17.054733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:17.134350 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa80b15-bdcb-4614-bf58-0795417d433e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:17.135370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:17.157066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1cbea8-821f-444d-ab8d-9479b93f7204 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:17.177709 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ad6d2-115a-4bae-9511-e280f3886c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:17.178988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ad6d2-115a-4bae-9511-e280f3886c05 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:17.179473 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ad6d2-115a-4bae-9511-e280f3886c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:17.180042 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ad6d2-115a-4bae-9511-e280f3886c05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:17.187511 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ad6d2-115a-4bae-9511-e280f3886c05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:17.189241 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ad6d2-115a-4bae-9511-e280f3886c05 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:17.205522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-9e9ad6d2-115a-4bae-9511-e280f3886c05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:17.205522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:17.205522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:17.205522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:17.205522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:17.205522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:17.205522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:17.205522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:17.205522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:17.205522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:17.205522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:17.205522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:17.205522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:17.205522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:17.205522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:17.206432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:17.207320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:17.207320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:17.207320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:17.207320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:17.207320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:17.207320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:17.207320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:17.207320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:17.209838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 291/599] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:17.217002 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ad6d2-115a-4bae-9511-e280f3886c05 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:17.228211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca7c41-be92-46fb-a74c-8762e8d349f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:17.228885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca7c41-be92-46fb-a74c-8762e8d349f1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:17.229181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca7c41-be92-46fb-a74c-8762e8d349f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:17.229448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca7c41-be92-46fb-a74c-8762e8d349f1 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:17.230294 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca7c41-be92-46fb-a74c-8762e8d349f1 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:17.230594 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca7c41-be92-46fb-a74c-8762e8d349f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:17.232030 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca7c41-be92-46fb-a74c-8762e8d349f1 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:17.266291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 309/600] 10.222.0.61 () {56 vars in 1139 bytes} [Thu Jul 14 10:01:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:17.271787 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca7c41-be92-46fb-a74c-8762e8d349f1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:17.386188 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ad6d2-115a-4bae-9511-e280f3886c05 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:17.432717 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:17.453064 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca7c41-be92-46fb-a74c-8762e8d349f1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:17.455614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde06f6e-5dd2-451f-b75a-b3791d6e5bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:17.456243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde06f6e-5dd2-451f-b75a-b3791d6e5bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:17.456585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde06f6e-5dd2-451f-b75a-b3791d6e5bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:17.456917 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde06f6e-5dd2-451f-b75a-b3791d6e5bb5 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:17.457796 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde06f6e-5dd2-451f-b75a-b3791d6e5bb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:17.458366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde06f6e-5dd2-451f-b75a-b3791d6e5bb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:17.460632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde06f6e-5dd2-451f-b75a-b3791d6e5bb5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:17.484053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 292/601] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:17.485590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d416a3-627c-4c6a-83e7-d2327af35ca9 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:17.485876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d416a3-627c-4c6a-83e7-d2327af35ca9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:17.486151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d416a3-627c-4c6a-83e7-d2327af35ca9 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:17.491269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 293/602] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:17.491798 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d416a3-627c-4c6a-83e7-d2327af35ca9 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:17.506162 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72bf546-dbc7-4289-83fd-73e444e32566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:17.506961 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bf546-dbc7-4289-83fd-73e444e32566 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:17.507241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bf546-dbc7-4289-83fd-73e444e32566 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:17.507510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72bf546-dbc7-4289-83fd-73e444e32566 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:17.519079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bf546-dbc7-4289-83fd-73e444e32566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:17.520530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72bf546-dbc7-4289-83fd-73e444e32566 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:17.530646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:17.540780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-f72bf546-dbc7-4289-83fd-73e444e32566 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:17.540780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:17.540780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:17.540780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:17.540780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:17.540780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:17.540780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:17.540780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:17.540780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:17.540780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:17.540780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:17.540780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:17.540780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:17.540780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:17.540780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:17.541722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:17.542615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:17.542615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:17.542615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:17.542615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:17.542615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 310/603] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:17.543608 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96210bc2-807e-45d1-bdd8-2cba820a7c13 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:17.544215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96210bc2-807e-45d1-bdd8-2cba820a7c13 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:17.544693 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96210bc2-807e-45d1-bdd8-2cba820a7c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:17.563990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-96210bc2-807e-45d1-bdd8-2cba820a7c13 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:17.565221 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-96210bc2-807e-45d1-bdd8-2cba820a7c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:17.635515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d416a3-627c-4c6a-83e7-d2327af35ca9 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:17.636331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 311/604] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:17.637342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96210bc2-807e-45d1-bdd8-2cba820a7c13 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:17.680891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c482b-065e-4059-92e7-4159b7ac9283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:17.682607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c482b-065e-4059-92e7-4159b7ac9283 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:17.683088 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c482b-065e-4059-92e7-4159b7ac9283 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:17.683953 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c482b-065e-4059-92e7-4159b7ac9283 None admin] PATH_INFO: `/v3/projects/57662256880f43018039715a755c087c/users/e3b5660cfd344dcd8abe14c88df686e8/roles/ba4088a289ea4b9a9aa9d9cd7a27e731` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:17.701728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c482b-065e-4059-92e7-4159b7ac9283 None admin] RBAC: Authorizing `identity:create_grant(project_id=57662256880f43018039715a755c087c, user_id=e3b5660cfd344dcd8abe14c88df686e8, role_id=ba4088a289ea4b9a9aa9d9cd7a27e731)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:17.703496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c482b-065e-4059-92e7-4159b7ac9283 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:17.721722 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 294/605] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:01:17 2022] PUT /identity/v3/projects/57662256880f43018039715a755c087c/users/e3b5660cfd344dcd8abe14c88df686e8/roles/ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:01:17.728953 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c482b-065e-4059-92e7-4159b7ac9283 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:17.771422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:17.781119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96210bc2-807e-45d1-bdd8-2cba820a7c13 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:17.849289 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:17.859195 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc4cdcc-506a-40c1-a71b-cb608a5f1039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:17.862240 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4cdcc-506a-40c1-a71b-cb608a5f1039 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:17.862545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4cdcc-506a-40c1-a71b-cb608a5f1039 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:17.862825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc4cdcc-506a-40c1-a71b-cb608a5f1039 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:17.863605 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4cdcc-506a-40c1-a71b-cb608a5f1039 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:17.863920 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4cdcc-506a-40c1-a71b-cb608a5f1039 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:17.865406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc4cdcc-506a-40c1-a71b-cb608a5f1039 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:17.900991 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 312/606] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:17.904765 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c482b-065e-4059-92e7-4159b7ac9283 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:17.912490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33f46c-8132-4898-a881-1de57cc540a0 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:17.916631 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33f46c-8132-4898-a881-1de57cc540a0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:17.916631 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33f46c-8132-4898-a881-1de57cc540a0 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:17.916631 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 313/607] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:01:17.916631 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33f46c-8132-4898-a881-1de57cc540a0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:17.960525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f11c82-692a-4e82-9491-570b3ed3be70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:17.961955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f11c82-692a-4e82-9491-570b3ed3be70 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:17.962234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f11c82-692a-4e82-9491-570b3ed3be70 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:17.962503 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f11c82-692a-4e82-9491-570b3ed3be70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:17.978337 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f11c82-692a-4e82-9491-570b3ed3be70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-78f11c82-692a-4e82-9491-570b3ed3be70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:17.987552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:17.988620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:17.989550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:17.989550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:17.989550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:17.989550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:17.989550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:17.989550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:17.989550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:17.989550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:17.989550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:17.989550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:17.989550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:17.989550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:17.993314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 295/608] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:17.995008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd7aff-b1bd-46db-9b29-fd78494c333d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:17.995506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd7aff-b1bd-46db-9b29-fd78494c333d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:17.995982 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fd7aff-b1bd-46db-9b29-fd78494c333d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:18.014098 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a6fd7aff-b1bd-46db-9b29-fd78494c333d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:18.015730 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a6fd7aff-b1bd-46db-9b29-fd78494c333d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:18.044407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33f46c-8132-4898-a881-1de57cc540a0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:18.088445 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a0b3b-5816-47e7-946a-24c33d36e33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:18.089288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a0b3b-5816-47e7-946a-24c33d36e33c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:18.089567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a0b3b-5816-47e7-946a-24c33d36e33c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:18.089837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a0b3b-5816-47e7-946a-24c33d36e33c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:18.091100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 296/609] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:18.096777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a0b3b-5816-47e7-946a-24c33d36e33c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:18.098005 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd7aff-b1bd-46db-9b29-fd78494c333d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:18.098233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a0b3b-5816-47e7-946a-24c33d36e33c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:18.115065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-4c7a0b3b-5816-47e7-946a-24c33d36e33c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:18.115065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:18.115065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:18.115065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:18.115065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:18.115065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:18.115065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:18.115065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:18.115065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:18.115065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:18.115065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:18.115065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:18.115065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:18.115065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:18.115065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:18.116135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:18.117101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:18.117101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:18.117101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:18.117101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:18.117101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:18.117101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:18.117101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:18.117101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:18.117101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 314/610] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:18.124737 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a0b3b-5816-47e7-946a-24c33d36e33c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:18.143964 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:18.190657 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:18.229755 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fd7aff-b1bd-46db-9b29-fd78494c333d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:18.265779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a0b3b-5816-47e7-946a-24c33d36e33c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:18.268843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666a183-335a-4346-a622-e8adde1ab1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:18.269562 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666a183-335a-4346-a622-e8adde1ab1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:18.269840 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666a183-335a-4346-a622-e8adde1ab1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:18.270117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2666a183-335a-4346-a622-e8adde1ab1a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:18.270929 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666a183-335a-4346-a622-e8adde1ab1a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:18.271342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666a183-335a-4346-a622-e8adde1ab1a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:18.272706 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2666a183-335a-4346-a622-e8adde1ab1a5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:18.282120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 297/611] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:18.282963 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666a183-335a-4346-a622-e8adde1ab1a5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:18.307752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9965ab3-6ecd-45a7-a520-5fb9f7dea41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:18.309715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9965ab3-6ecd-45a7-a520-5fb9f7dea41a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:18.310107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9965ab3-6ecd-45a7-a520-5fb9f7dea41a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:18.310775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9965ab3-6ecd-45a7-a520-5fb9f7dea41a None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:18.312169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9965ab3-6ecd-45a7-a520-5fb9f7dea41a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:18.312828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9965ab3-6ecd-45a7-a520-5fb9f7dea41a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:18.315421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9965ab3-6ecd-45a7-a520-5fb9f7dea41a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:18.331847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 315/612] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:18.333034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9965ab3-6ecd-45a7-a520-5fb9f7dea41a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:18.374362 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:18.392326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2666a183-335a-4346-a622-e8adde1ab1a5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:18.426970 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa2194-2952-4d4f-8332-7683bd8edc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:18.427761 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa2194-2952-4d4f-8332-7683bd8edc30 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:18.428039 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa2194-2952-4d4f-8332-7683bd8edc30 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:18.428320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fa2194-2952-4d4f-8332-7683bd8edc30 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:18.439215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fa2194-2952-4d4f-8332-7683bd8edc30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-86fa2194-2952-4d4f-8332-7683bd8edc30 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:18.447480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:18.448408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:18.449668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:18.449668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:18.449668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:18.449668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:18.449668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:18.449668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:18.449668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:18.449668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:18.449668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:18.449668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:18.449668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:18.449668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:18.453671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 298/613] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:18.454334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa2194-2952-4d4f-8332-7683bd8edc30 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:18.469743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9965ab3-6ecd-45a7-a520-5fb9f7dea41a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:18.484396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:18.520378 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269fc20-1d6a-4985-a3a9-7d3646619830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:18.521207 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269fc20-1d6a-4985-a3a9-7d3646619830 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:18.521483 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269fc20-1d6a-4985-a3a9-7d3646619830 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:18.521754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1269fc20-1d6a-4985-a3a9-7d3646619830 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:18.532881 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269fc20-1d6a-4985-a3a9-7d3646619830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:18.534339 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1269fc20-1d6a-4985-a3a9-7d3646619830 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:18.552013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-1269fc20-1d6a-4985-a3a9-7d3646619830 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:18.552013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:18.552013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:18.552013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:18.552013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:18.552013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:18.552013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:18.552013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:18.552013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:18.552013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:18.552013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:18.552013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:18.552013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:18.552013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:18.552013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:18.553007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:18.553903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:18.553903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:18.553903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:18.553903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:18.553903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 316/614] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:18.554661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269fc20-1d6a-4985-a3a9-7d3646619830 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:18.570326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fa2194-2952-4d4f-8332-7683bd8edc30 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:18.607875 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4439b58-4ffa-4b61-8fe4-9193918eb8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:18.609157 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4439b58-4ffa-4b61-8fe4-9193918eb8dd None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:18.609669 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4439b58-4ffa-4b61-8fe4-9193918eb8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:18.610238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4439b58-4ffa-4b61-8fe4-9193918eb8dd None admin] PATH_INFO: `/v3/projects/57662256880f43018039715a755c087c/users/cd6d19c4406b4b228065b98a9a074b3a/roles/ba4088a289ea4b9a9aa9d9cd7a27e731` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:18.626680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4439b58-4ffa-4b61-8fe4-9193918eb8dd None admin] RBAC: Authorizing `identity:create_grant(project_id=57662256880f43018039715a755c087c, user_id=cd6d19c4406b4b228065b98a9a074b3a, role_id=ba4088a289ea4b9a9aa9d9cd7a27e731)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:18.628590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4439b58-4ffa-4b61-8fe4-9193918eb8dd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:18.631581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:18.646036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 299/615] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:01:18 2022] PUT /identity/v3/projects/57662256880f43018039715a755c087c/users/cd6d19c4406b4b228065b98a9a074b3a/roles/ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:01:18.647318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4439b58-4ffa-4b61-8fe4-9193918eb8dd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:18.667092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1269fc20-1d6a-4985-a3a9-7d3646619830 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:18.706685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4abb3-24cb-41dd-8135-7d5adce9d86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:18.707334 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4abb3-24cb-41dd-8135-7d5adce9d86a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:18.707616 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4abb3-24cb-41dd-8135-7d5adce9d86a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:18.707889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4abb3-24cb-41dd-8135-7d5adce9d86a None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:18.708818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4abb3-24cb-41dd-8135-7d5adce9d86a None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:18.709282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4abb3-24cb-41dd-8135-7d5adce9d86a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:18.710913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4abb3-24cb-41dd-8135-7d5adce9d86a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:18.720984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 317/616] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:18.730371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4abb3-24cb-41dd-8135-7d5adce9d86a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:18.768304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4439b58-4ffa-4b61-8fe4-9193918eb8dd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:18.771071 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:18.810016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acd803-7feb-4551-8e78-2379f6e01ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:18.810979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acd803-7feb-4551-8e78-2379f6e01ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:18.811420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acd803-7feb-4551-8e78-2379f6e01ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:18.811914 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64acd803-7feb-4551-8e78-2379f6e01ec5 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:18.812906 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64acd803-7feb-4551-8e78-2379f6e01ec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:18.813386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64acd803-7feb-4551-8e78-2379f6e01ec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:18.814908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64acd803-7feb-4551-8e78-2379f6e01ec5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:18.829652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 300/617] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:18.839374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acd803-7feb-4551-8e78-2379f6e01ec5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:18.905447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:18.933258 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4abb3-24cb-41dd-8135-7d5adce9d86a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:18.963991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64acd803-7feb-4551-8e78-2379f6e01ec5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:18.980054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a01c7-545a-43ff-aa6d-857642db3b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:18.981346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a01c7-545a-43ff-aa6d-857642db3b89 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:18.981632 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a01c7-545a-43ff-aa6d-857642db3b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:18.982270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408a01c7-545a-43ff-aa6d-857642db3b89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:18.996695 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a01c7-545a-43ff-aa6d-857642db3b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:18.999471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408a01c7-545a-43ff-aa6d-857642db3b89 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:19.004302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51322a9e-8255-47b3-a7b9-0240cac937ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:19.005246 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51322a9e-8255-47b3-a7b9-0240cac937ae None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:19.005626 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51322a9e-8255-47b3-a7b9-0240cac937ae None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:19.006000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51322a9e-8255-47b3-a7b9-0240cac937ae None admin] PATH_INFO: `/v3/users/swift` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:19.015312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51322a9e-8255-47b3-a7b9-0240cac937ae None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:19.016868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51322a9e-8255-47b3-a7b9-0240cac937ae None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:19.020183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-408a01c7-545a-43ff-aa6d-857642db3b89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:19.020183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:19.020183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:19.020183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:19.020183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:19.020183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:19.020183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:19.020183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:19.020183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:19.020183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:19.020183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:19.020183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:19.020183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:19.020183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:19.020183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:19.021268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:19.022473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:19.022473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:19.022473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:19.022473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:19.022473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 318/618] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:19.029994 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a01c7-545a-43ff-aa6d-857642db3b89 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-51322a9e-8255-47b3-a7b9-0240cac937ae None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:19.038195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:19.039575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:19.040876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:19.040876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:19.040876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:19.040876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:19.040876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:19.040876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:19.040876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:19.040876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:19.040876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:01:19.040876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:19.041563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 301/619] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:01:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:19.049520 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51322a9e-8255-47b3-a7b9-0240cac937ae None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:19.191317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:19.204709 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51322a9e-8255-47b3-a7b9-0240cac937ae None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:19.216838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408a01c7-545a-43ff-aa6d-857642db3b89 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:19.220020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:19.254688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199f17b-c1d8-4103-af71-8575a112a4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:19.255378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199f17b-c1d8-4103-af71-8575a112a4ab None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:19.255659 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199f17b-c1d8-4103-af71-8575a112a4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:19.255931 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199f17b-c1d8-4103-af71-8575a112a4ab None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:19.256784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199f17b-c1d8-4103-af71-8575a112a4ab None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:19.257121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199f17b-c1d8-4103-af71-8575a112a4ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:19.258495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3271ab-ccca-419d-9553-69377ad6ea27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:19.259095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199f17b-c1d8-4103-af71-8575a112a4ab None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:19.259598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3271ab-ccca-419d-9553-69377ad6ea27 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:19.260094 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3271ab-ccca-419d-9553-69377ad6ea27 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:19.260568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3271ab-ccca-419d-9553-69377ad6ea27 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:19.261519 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3271ab-ccca-419d-9553-69377ad6ea27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:19.262027 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3271ab-ccca-419d-9553-69377ad6ea27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:19.263570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3271ab-ccca-419d-9553-69377ad6ea27 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:19.277039 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 319/620] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:19.286933 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3271ab-ccca-419d-9553-69377ad6ea27 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:19.287107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 302/621] 10.222.0.61 () {56 vars in 1135 bytes} [Thu Jul 14 10:01:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:19.292029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199f17b-c1d8-4103-af71-8575a112a4ab None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:19.415962 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199f17b-c1d8-4103-af71-8575a112a4ab None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:19.446930 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3271ab-ccca-419d-9553-69377ad6ea27 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:19.454749 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c1530a-eb2d-4d27-8afa-39307d545d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:19.455538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1530a-eb2d-4d27-8afa-39307d545d3b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:19.455861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1530a-eb2d-4d27-8afa-39307d545d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:19.456134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1530a-eb2d-4d27-8afa-39307d545d3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:19.466981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1530a-eb2d-4d27-8afa-39307d545d3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:19.468407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c1530a-eb2d-4d27-8afa-39307d545d3b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:19.487202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd66686-042f-4fea-aca5-5fad7f325ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:19.487380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-38c1530a-eb2d-4d27-8afa-39307d545d3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:19.487380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:19.487380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:19.487380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:19.487380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:19.487380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:19.487380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:19.487380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:19.487380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:19.487380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:19.487380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:19.487380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:19.487380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:19.487380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:19.487380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:19.488251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:19.489167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:19.489167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:19.489167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:19.489167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:19.489414 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66686-042f-4fea-aca5-5fad7f325ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:19.489414 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66686-042f-4fea-aca5-5fad7f325ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:19.489414 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66686-042f-4fea-aca5-5fad7f325ca2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:19.493631 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 303/622] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:19.494859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37842ae6-786b-40f4-8e07-2a57c1a8e97a None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:19.495138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37842ae6-786b-40f4-8e07-2a57c1a8e97a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:19.495392 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:19.495531 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37842ae6-786b-40f4-8e07-2a57c1a8e97a None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:19.501183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 304/623] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:19.501183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37842ae6-786b-40f4-8e07-2a57c1a8e97a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:19.504900 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd66686-042f-4fea-aca5-5fad7f325ca2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:19.506448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd66686-042f-4fea-aca5-5fad7f325ca2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-3bd66686-042f-4fea-aca5-5fad7f325ca2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:19.521982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:19.523214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:19.524315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:19.524315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:19.524315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:19.524315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:19.524315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:19.524315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:19.524315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:19.524315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:19.524315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:01:19.524315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:19.524315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 320/624] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Jul 14 10:01:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:19.525010 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28566e6-9d12-4dbc-814c-646fcbfc09e8 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:19.525270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28566e6-9d12-4dbc-814c-646fcbfc09e8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:19.525509 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28566e6-9d12-4dbc-814c-646fcbfc09e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:19.535038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:19.543120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e28566e6-9d12-4dbc-814c-646fcbfc09e8 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:19.544360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e28566e6-9d12-4dbc-814c-646fcbfc09e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:19.620950 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 321/625] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:19.620950 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28566e6-9d12-4dbc-814c-646fcbfc09e8 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:19.630423 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37842ae6-786b-40f4-8e07-2a57c1a8e97a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:19.687816 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe61345-1b61-498f-b968-00d4a214bdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:19.689028 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe61345-1b61-498f-b968-00d4a214bdcf None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:19.689483 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe61345-1b61-498f-b968-00d4a214bdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:19.690146 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe61345-1b61-498f-b968-00d4a214bdcf None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:19.691896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe61345-1b61-498f-b968-00d4a214bdcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:19.692384 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe61345-1b61-498f-b968-00d4a214bdcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:19.693981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe61345-1b61-498f-b968-00d4a214bdcf None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:19.711182 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 305/626] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:19.712237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe61345-1b61-498f-b968-00d4a214bdcf None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:19.728241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28566e6-9d12-4dbc-814c-646fcbfc09e8 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:19.755763 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:19.766814 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09267a94-895f-4da3-8da9-ee024732612c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:19.767537 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09267a94-895f-4da3-8da9-ee024732612c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:19.767817 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09267a94-895f-4da3-8da9-ee024732612c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:19.768394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09267a94-895f-4da3-8da9-ee024732612c None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:19.768940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09267a94-895f-4da3-8da9-ee024732612c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:19.769254 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09267a94-895f-4da3-8da9-ee024732612c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:19.770769 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09267a94-895f-4da3-8da9-ee024732612c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:19.789285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 322/627] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Jul 14 10:01:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:19.790092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09267a94-895f-4da3-8da9-ee024732612c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:19.825272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:19.843143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe61345-1b61-498f-b968-00d4a214bdcf None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:19.880599 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14570efa-e662-4928-993d-593b1691351a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:19.881466 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14570efa-e662-4928-993d-593b1691351a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:19.881750 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14570efa-e662-4928-993d-593b1691351a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:19.882023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14570efa-e662-4928-993d-593b1691351a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:19.892837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14570efa-e662-4928-993d-593b1691351a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-14570efa-e662-4928-993d-593b1691351a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:19.900588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:19.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:19.902781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:19.902781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:19.902781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:19.902781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:19.902781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:19.902781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:19.902781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:19.902781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:19.902781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:19.902781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:19.902781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:19.902781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:19.906637 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 306/628] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:19.907486 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14570efa-e662-4928-993d-593b1691351a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:19.924892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09267a94-895f-4da3-8da9-ee024732612c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:19.966255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff832988-2e7d-4efa-abbf-81cda93dc2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:19.967047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff832988-2e7d-4efa-abbf-81cda93dc2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:19.967329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff832988-2e7d-4efa-abbf-81cda93dc2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:19.967599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff832988-2e7d-4efa-abbf-81cda93dc2b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:19.974505 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff832988-2e7d-4efa-abbf-81cda93dc2b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:19.975967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff832988-2e7d-4efa-abbf-81cda93dc2b8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:19.990169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-ff832988-2e7d-4efa-abbf-81cda93dc2b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:19.990169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:19.990169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:19.990169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:19.990169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:19.990169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:19.990169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:19.990169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:19.990169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:19.990169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:19.990169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:19.990169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:19.990169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:19.990169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:19.990169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:19.991285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:19.992798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:19.992798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:19.992798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:19.992798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:19.992798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:19.992798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:19.992798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:19.992798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:19.992798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 323/629] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:19.993769 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff832988-2e7d-4efa-abbf-81cda93dc2b8 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:20.020036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14570efa-e662-4928-993d-593b1691351a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:20.045940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:20.059900 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b884d3-ada8-4124-95ef-e38d857e3b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:20.060573 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b884d3-ada8-4124-95ef-e38d857e3b56 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:20.060830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b884d3-ada8-4124-95ef-e38d857e3b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:20.061125 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b884d3-ada8-4124-95ef-e38d857e3b56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:20.077200 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:20.106408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff832988-2e7d-4efa-abbf-81cda93dc2b8 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:20.135704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b884d3-ada8-4124-95ef-e38d857e3b56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:20.137193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b884d3-ada8-4124-95ef-e38d857e3b56 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:20.139827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b013112-7d1e-40fd-81a3-f07d16267e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:20.140485 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b013112-7d1e-40fd-81a3-f07d16267e72 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:20.140763 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b013112-7d1e-40fd-81a3-f07d16267e72 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:20.141063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b013112-7d1e-40fd-81a3-f07d16267e72 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:20.141808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b013112-7d1e-40fd-81a3-f07d16267e72 None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:20.142113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b013112-7d1e-40fd-81a3-f07d16267e72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:20.143440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b013112-7d1e-40fd-81a3-f07d16267e72 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:20.156592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 324/630] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:20.156592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847904e-aaae-4fa0-ae44-07f892b5cd7f None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:20.156592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847904e-aaae-4fa0-ae44-07f892b5cd7f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:20.156592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847904e-aaae-4fa0-ae44-07f892b5cd7f None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:20.156592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 325/631] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:20.157514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847904e-aaae-4fa0-ae44-07f892b5cd7f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:20.258104 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 307/632] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:01:20.258104 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b884d3-ada8-4124-95ef-e38d857e3b56 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:20.270953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847904e-aaae-4fa0-ae44-07f892b5cd7f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:20.279611 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:20.303618 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:20.312073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2a225-ac19-4649-b5b3-e8ce593f76a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:20.312733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2a225-ac19-4649-b5b3-e8ce593f76a0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:20.313041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2a225-ac19-4649-b5b3-e8ce593f76a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:20.313314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2a225-ac19-4649-b5b3-e8ce593f76a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:20.314101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2a225-ac19-4649-b5b3-e8ce593f76a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:20.314408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2a225-ac19-4649-b5b3-e8ce593f76a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:20.315903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2a225-ac19-4649-b5b3-e8ce593f76a0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:20.327706 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 326/633] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:20.328728 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5b077-190c-4b3a-ba39-d7a1414e6e3f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:20.329045 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5b077-190c-4b3a-ba39-d7a1414e6e3f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:20.329314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5b077-190c-4b3a-ba39-d7a1414e6e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:20.345593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c4a5b077-190c-4b3a-ba39-d7a1414e6e3f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:20.346766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c4a5b077-190c-4b3a-ba39-d7a1414e6e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:20.370640 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b884d3-ada8-4124-95ef-e38d857e3b56 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:20.407676 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccaa37-2629-4bb1-b8d3-c7820ccdf60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:20.408485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccaa37-2629-4bb1-b8d3-c7820ccdf60b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:20.408761 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccaa37-2629-4bb1-b8d3-c7820ccdf60b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:20.409059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ccaa37-2629-4bb1-b8d3-c7820ccdf60b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:20.424953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 327/634] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:20.424953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5b077-190c-4b3a-ba39-d7a1414e6e3f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:20.430240 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccaa37-2629-4bb1-b8d3-c7820ccdf60b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:20.431758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ccaa37-2629-4bb1-b8d3-c7820ccdf60b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:20.450787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-18ccaa37-2629-4bb1-b8d3-c7820ccdf60b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:20.450787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:20.450787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:20.450787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:20.450787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:20.450787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:20.450787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:20.450787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:20.450787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:20.450787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:20.450787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:20.450787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:20.450787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:20.450787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:20.450787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:20.451715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:20.452609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:20.452609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:20.452609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:20.452609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:20.452609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 308/635] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:20.453555 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccaa37-2629-4bb1-b8d3-c7820ccdf60b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:20.528418 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:20.535262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5b077-190c-4b3a-ba39-d7a1414e6e3f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:20.551086 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:20.558782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ccaa37-2629-4bb1-b8d3-c7820ccdf60b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:20.572266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5000e-f3c2-46c2-aed6-0f76c5a75c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:20.573357 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5000e-f3c2-46c2-aed6-0f76c5a75c6a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:20.573814 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5000e-f3c2-46c2-aed6-0f76c5a75c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:20.574268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a5000e-f3c2-46c2-aed6-0f76c5a75c6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:20.585162 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5000e-f3c2-46c2-aed6-0f76c5a75c6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:20.586620 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a5000e-f3c2-46c2-aed6-0f76c5a75c6a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:20.594367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d3bab-dc10-42e8-b5bc-7fd715e22c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:20.595220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d3bab-dc10-42e8-b5bc-7fd715e22c0f None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:20.595496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d3bab-dc10-42e8-b5bc-7fd715e22c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:20.595767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600d3bab-dc10-42e8-b5bc-7fd715e22c0f None admin] PATH_INFO: `/v3/projects/57662256880f43018039715a755c087c/users/d5fa502e8b014c719fcece0a130fdd7e/roles/ba4088a289ea4b9a9aa9d9cd7a27e731` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:20.604909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-d3a5000e-f3c2-46c2-aed6-0f76c5a75c6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:20.604909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:20.604909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:20.604909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:20.604909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:20.604909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:20.604909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:20.604909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:20.604909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:20.604909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:20.604909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:20.604909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:20.604909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:20.604909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:20.604909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:20.605819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:20.606702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:20.606702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:20.606702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:20.606702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:20.606702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 328/636] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:20.606995 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5000e-f3c2-46c2-aed6-0f76c5a75c6a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:20.615513 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d3bab-dc10-42e8-b5bc-7fd715e22c0f None admin] RBAC: Authorizing `identity:create_grant(project_id=57662256880f43018039715a755c087c, user_id=d5fa502e8b014c719fcece0a130fdd7e, role_id=ba4088a289ea4b9a9aa9d9cd7a27e731)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:20.617541 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600d3bab-dc10-42e8-b5bc-7fd715e22c0f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:20.631692 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 309/637] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:01:20 2022] PUT /identity/v3/projects/57662256880f43018039715a755c087c/users/d5fa502e8b014c719fcece0a130fdd7e/roles/ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:01:20.633150 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d3bab-dc10-42e8-b5bc-7fd715e22c0f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:20.721554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a5000e-f3c2-46c2-aed6-0f76c5a75c6a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:20.764023 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aed105-250c-4cca-9493-3413aebd9577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:20.765373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aed105-250c-4cca-9493-3413aebd9577 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:20.765751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aed105-250c-4cca-9493-3413aebd9577 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:20.766124 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16aed105-250c-4cca-9493-3413aebd9577 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:20.784396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16aed105-250c-4cca-9493-3413aebd9577 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:20.786068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600d3bab-dc10-42e8-b5bc-7fd715e22c0f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-16aed105-250c-4cca-9493-3413aebd9577 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:20.794800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:20.795890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:20.796777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:20.796777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:20.796777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:20.796777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:20.796777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:20.796777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:20.796777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:20.796777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:20.796777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:20.796777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:20.796777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:20.796777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:20.803742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 329/638] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:20.805513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aed105-250c-4cca-9493-3413aebd9577 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:20.818008 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:20.830598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e9bde-fe31-4c5a-8117-0d1b74f15fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:20.831269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e9bde-fe31-4c5a-8117-0d1b74f15fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:20.831546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e9bde-fe31-4c5a-8117-0d1b74f15fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:20.831817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e9bde-fe31-4c5a-8117-0d1b74f15fc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:20.832546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e9bde-fe31-4c5a-8117-0d1b74f15fc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:20.832849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e9bde-fe31-4c5a-8117-0d1b74f15fc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:20.834246 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e9bde-fe31-4c5a-8117-0d1b74f15fc5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:20.846720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 310/639] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:20.847533 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e9bde-fe31-4c5a-8117-0d1b74f15fc5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:20.857444 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:20.930331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16aed105-250c-4cca-9493-3413aebd9577 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:20.965692 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da3e42-a2c5-436a-a2ab-a71962ff20be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:20.966643 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da3e42-a2c5-436a-a2ab-a71962ff20be None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:20.967145 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da3e42-a2c5-436a-a2ab-a71962ff20be None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:20.967565 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da3e42-a2c5-436a-a2ab-a71962ff20be None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:20.968432 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da3e42-a2c5-436a-a2ab-a71962ff20be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:20.968902 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da3e42-a2c5-436a-a2ab-a71962ff20be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:20.970491 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22da3e42-a2c5-436a-a2ab-a71962ff20be None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:20.971314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e9bde-fe31-4c5a-8117-0d1b74f15fc5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:20.983876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 330/640] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:20.984962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da3e42-a2c5-436a-a2ab-a71962ff20be None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:21.013829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc10a318-03c8-4946-a39c-9f8a2e46acd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:21.014492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10a318-03c8-4946-a39c-9f8a2e46acd9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:21.014768 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10a318-03c8-4946-a39c-9f8a2e46acd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:21.015032 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc10a318-03c8-4946-a39c-9f8a2e46acd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:21.015881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10a318-03c8-4946-a39c-9f8a2e46acd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:21.016292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10a318-03c8-4946-a39c-9f8a2e46acd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:21.017688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc10a318-03c8-4946-a39c-9f8a2e46acd9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:21.029292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 311/641] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:21.029292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55d15c-8455-4bf5-bd86-96a854f138c8 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:21.029292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55d15c-8455-4bf5-bd86-96a854f138c8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:21.029292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55d15c-8455-4bf5-bd86-96a854f138c8 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:21.030126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 312/642] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:21.031687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55d15c-8455-4bf5-bd86-96a854f138c8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:21.055686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:21.091567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22da3e42-a2c5-436a-a2ab-a71962ff20be None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:21.109941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:21.126244 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba26fa-3982-4be2-9bdd-bfea6dc6e43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:21.127043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba26fa-3982-4be2-9bdd-bfea6dc6e43a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:21.127329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba26fa-3982-4be2-9bdd-bfea6dc6e43a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:21.127601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeba26fa-3982-4be2-9bdd-bfea6dc6e43a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:21.134943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba26fa-3982-4be2-9bdd-bfea6dc6e43a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:21.136367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeba26fa-3982-4be2-9bdd-bfea6dc6e43a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:21.146615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55d15c-8455-4bf5-bd86-96a854f138c8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-aeba26fa-3982-4be2-9bdd-bfea6dc6e43a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:21.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:21.152268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:21.153261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:21.153261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:21.153261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:21.153261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:21.153261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:21.153261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:21.153261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:21.153261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:21.153261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:01:21.153261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:21.157075 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 331/643] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:01:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:21.157908 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba26fa-3982-4be2-9bdd-bfea6dc6e43a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:21.186727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a344f237-8ccc-4cda-8ca7-ae25e19032ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:21.187465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344f237-8ccc-4cda-8ca7-ae25e19032ec None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:21.187742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344f237-8ccc-4cda-8ca7-ae25e19032ec None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:21.188006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344f237-8ccc-4cda-8ca7-ae25e19032ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:21.195561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344f237-8ccc-4cda-8ca7-ae25e19032ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:21.197004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a344f237-8ccc-4cda-8ca7-ae25e19032ec None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:21.211057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-a344f237-8ccc-4cda-8ca7-ae25e19032ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:21.211057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:21.211057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:21.211057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:21.211057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:21.211057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:21.211057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:21.211057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:21.211057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:21.211057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:21.211057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:21.211057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:21.211057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:21.211057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:21.211057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:21.211956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:21.213064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:21.213064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:21.213064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:21.213064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:21.213064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:21.213064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:21.213064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:21.213064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:21.216914 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 313/644] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:21.218187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a6181-6318-42d7-ad93-5f0743b53c89 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:21.218468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a6181-6318-42d7-ad93-5f0743b53c89 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:21.218740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a6181-6318-42d7-ad93-5f0743b53c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:21.235373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8e0a6181-6318-42d7-ad93-5f0743b53c89 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:21.237421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8e0a6181-6318-42d7-ad93-5f0743b53c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:21.265481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeba26fa-3982-4be2-9bdd-bfea6dc6e43a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:21.298378 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:21.307737 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0e1ff-8efe-4196-ab61-21ebf5882616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:21.309131 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 314/645] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:21.309652 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0e1ff-8efe-4196-ab61-21ebf5882616 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:21.309974 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a6181-6318-42d7-ad93-5f0743b53c89 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:21.310224 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0e1ff-8efe-4196-ab61-21ebf5882616 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:21.310622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c0e1ff-8efe-4196-ab61-21ebf5882616 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:21.331754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0e1ff-8efe-4196-ab61-21ebf5882616 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:21.333880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c0e1ff-8efe-4196-ab61-21ebf5882616 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:21.352401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-a0c0e1ff-8efe-4196-ab61-21ebf5882616 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:21.352401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:21.352401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:21.352401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:21.352401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:21.352401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:21.352401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:21.352401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:21.352401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:21.352401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:21.352401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:21.352401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:21.352401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:21.352401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:21.352401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:21.353355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:21.354242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:21.354242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:21.354242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:21.354242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:21.354242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 332/646] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:21.354872 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0e1ff-8efe-4196-ab61-21ebf5882616 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:21.377012 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:21.423390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a6181-6318-42d7-ad93-5f0743b53c89 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:21.457860 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c0e1ff-8efe-4196-ab61-21ebf5882616 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:21.461275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dc5c8-dc46-4f54-9979-26b07fc56c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:21.461412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dc5c8-dc46-4f54-9979-26b07fc56c37 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:21.461667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dc5c8-dc46-4f54-9979-26b07fc56c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:21.461941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24dc5c8-dc46-4f54-9979-26b07fc56c37 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:21.462779 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24dc5c8-dc46-4f54-9979-26b07fc56c37 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:21.463086 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24dc5c8-dc46-4f54-9979-26b07fc56c37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:21.464504 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24dc5c8-dc46-4f54-9979-26b07fc56c37 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:21.482667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 315/647] 10.222.0.61 () {56 vars in 1137 bytes} [Thu Jul 14 10:01:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:21.483538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dc5c8-dc46-4f54-9979-26b07fc56c37 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:21.497213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872999f-aab7-4afc-93ed-f3faf6175407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:21.497886 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872999f-aab7-4afc-93ed-f3faf6175407 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:21.498146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872999f-aab7-4afc-93ed-f3faf6175407 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:21.498413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8872999f-aab7-4afc-93ed-f3faf6175407 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:21.499195 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872999f-aab7-4afc-93ed-f3faf6175407 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:21.499500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872999f-aab7-4afc-93ed-f3faf6175407 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:21.500908 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8872999f-aab7-4afc-93ed-f3faf6175407 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:21.518583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 333/648] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:21.518583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872999f-aab7-4afc-93ed-f3faf6175407 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:21.558828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:21.592656 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24dc5c8-dc46-4f54-9979-26b07fc56c37 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:21.609470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:21.622177 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8872999f-aab7-4afc-93ed-f3faf6175407 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:21.628245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403baf30-b70c-4d45-9643-2fc8b9f8ee64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:21.629077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403baf30-b70c-4d45-9643-2fc8b9f8ee64 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:21.629405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403baf30-b70c-4d45-9643-2fc8b9f8ee64 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:21.630009 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403baf30-b70c-4d45-9643-2fc8b9f8ee64 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:21.641174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403baf30-b70c-4d45-9643-2fc8b9f8ee64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-403baf30-b70c-4d45-9643-2fc8b9f8ee64 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:21.649733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:21.650671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:21.651518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:21.651518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:21.651518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:21.651518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:21.651518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:21.651518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:21.651518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:21.651518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:21.651518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:21.651518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:21.651518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:21.651518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:21.651518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 316/649] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:21.652234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403baf30-b70c-4d45-9643-2fc8b9f8ee64 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:21.664202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680341e7-49dd-4f1c-b8f5-e73c34822503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:21.664863 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680341e7-49dd-4f1c-b8f5-e73c34822503 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:21.665169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680341e7-49dd-4f1c-b8f5-e73c34822503 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:21.665443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680341e7-49dd-4f1c-b8f5-e73c34822503 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:21.666162 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680341e7-49dd-4f1c-b8f5-e73c34822503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:21.666465 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680341e7-49dd-4f1c-b8f5-e73c34822503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:21.667813 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680341e7-49dd-4f1c-b8f5-e73c34822503 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:21.680310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 334/650] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:21.680969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680341e7-49dd-4f1c-b8f5-e73c34822503 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:21.765338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403baf30-b70c-4d45-9643-2fc8b9f8ee64 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:21.792608 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:21.796043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680341e7-49dd-4f1c-b8f5-e73c34822503 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:21.802804 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00caa71-49ff-4f20-ba26-f78917b44086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:21.803479 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00caa71-49ff-4f20-ba26-f78917b44086 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:21.804299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00caa71-49ff-4f20-ba26-f78917b44086 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:21.804577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00caa71-49ff-4f20-ba26-f78917b44086 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:21.833141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cd22e-f165-41ff-aa34-c0ea68533ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:21.833803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cd22e-f165-41ff-aa34-c0ea68533ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:21.834093 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cd22e-f165-41ff-aa34-c0ea68533ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:21.834369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22cd22e-f165-41ff-aa34-c0ea68533ba6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:21.835370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cd22e-f165-41ff-aa34-c0ea68533ba6 None admin] RBAC: Adding query filter params (role.id=ba4088a289ea4b9a9aa9d9cd7a27e731, scope.project.id=57662256880f43018039715a755c087c, user.id=e9f4efa3e2f343c2ac858a277601a7df) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:21.835683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cd22e-f165-41ff-aa34-c0ea68533ba6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:21.837141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22cd22e-f165-41ff-aa34-c0ea68533ba6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:21.843290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 335/651] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:21 2022] GET /identity/v3/role_assignments?user.id=e9f4efa3e2f343c2ac858a277601a7df&scope.project.id=57662256880f43018039715a755c087c&role.id=ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:21.844110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cd22e-f165-41ff-aa34-c0ea68533ba6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:21.862745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:21.897262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00caa71-49ff-4f20-ba26-f78917b44086 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:21.900216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00caa71-49ff-4f20-ba26-f78917b44086 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:21.961356 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22cd22e-f165-41ff-aa34-c0ea68533ba6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:22.001312 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85df561-dd3e-493b-9f0e-604b7e2423f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:22.001988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85df561-dd3e-493b-9f0e-604b7e2423f2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:22.002267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85df561-dd3e-493b-9f0e-604b7e2423f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:22.002540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85df561-dd3e-493b-9f0e-604b7e2423f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:22.003301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85df561-dd3e-493b-9f0e-604b7e2423f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:22.003607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85df561-dd3e-493b-9f0e-604b7e2423f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:22.004987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85df561-dd3e-493b-9f0e-604b7e2423f2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:22.014809 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 336/652] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:22.015602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85df561-dd3e-493b-9f0e-604b7e2423f2 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:22.041736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 317/653] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:01:22.042590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00caa71-49ff-4f20-ba26-f78917b44086 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:22.050432 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:22.137392 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:22.143821 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85df561-dd3e-493b-9f0e-604b7e2423f2 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:22.165991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00caa71-49ff-4f20-ba26-f78917b44086 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:22.190332 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeecc0-b2b8-49d2-9671-a9b09b1f5b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:22.191507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeecc0-b2b8-49d2-9671-a9b09b1f5b65 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:22.192023 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeecc0-b2b8-49d2-9671-a9b09b1f5b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:22.192513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eeecc0-b2b8-49d2-9671-a9b09b1f5b65 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:22.201166 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eeecc0-b2b8-49d2-9671-a9b09b1f5b65 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:22.202957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eeecc0-b2b8-49d2-9671-a9b09b1f5b65 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:22.213945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e653c-7d7d-4b2d-8c1a-29f23e92a895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:22.214740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e653c-7d7d-4b2d-8c1a-29f23e92a895 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:22.215020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e653c-7d7d-4b2d-8c1a-29f23e92a895 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:22.215293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e653c-7d7d-4b2d-8c1a-29f23e92a895 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-c7eeecc0-b2b8-49d2-9671-a9b09b1f5b65 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:22.219333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:22.220421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:22.221828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:22.221828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:22.221828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:22.221828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:22.221828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:22.221828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:22.221828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:22.221828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:22.221828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 14 10:01:22.221828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:22.228951 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e653c-7d7d-4b2d-8c1a-29f23e92a895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:22.228951 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e653c-7d7d-4b2d-8c1a-29f23e92a895 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:22.229379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 337/654] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:01:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:22.230835 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeecc0-b2b8-49d2-9671-a9b09b1f5b65 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:22.259415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-652e653c-7d7d-4b2d-8c1a-29f23e92a895 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:22.259415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:22.259415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:22.259415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:22.259415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:22.259415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:22.259415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:22.259415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:22.259415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:22.259415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:22.259415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:22.259415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:22.259415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:22.259415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:22.259415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:22.260360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:22.261704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:22.261704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:22.261704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:22.261704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:22.265066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 318/655] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:22.266174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e653c-7d7d-4b2d-8c1a-29f23e92a895 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:22.352716 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:22.375210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eeecc0-b2b8-49d2-9671-a9b09b1f5b65 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:22.377735 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e653c-7d7d-4b2d-8c1a-29f23e92a895 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:22.406745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:22.416415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499e40f-8db7-442d-917c-e79034b10d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:22.417245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499e40f-8db7-442d-917c-e79034b10d20 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:22.417728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499e40f-8db7-442d-917c-e79034b10d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:22.418024 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499e40f-8db7-442d-917c-e79034b10d20 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:22.418975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499e40f-8db7-442d-917c-e79034b10d20 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:22.419385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499e40f-8db7-442d-917c-e79034b10d20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:22.420913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499e40f-8db7-442d-917c-e79034b10d20 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:22.423313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915805c-3879-49d7-8b24-1edd170cac4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:22.425277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915805c-3879-49d7-8b24-1edd170cac4a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:22.425731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915805c-3879-49d7-8b24-1edd170cac4a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:22.426170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6915805c-3879-49d7-8b24-1edd170cac4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:22.440903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915805c-3879-49d7-8b24-1edd170cac4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:22.442647 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6915805c-3879-49d7-8b24-1edd170cac4a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:22.443049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 319/656] 10.222.0.61 () {56 vars in 1137 bytes} [Thu Jul 14 10:01:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:22.444197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499e40f-8db7-442d-917c-e79034b10d20 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:22.468241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-6915805c-3879-49d7-8b24-1edd170cac4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:22.468241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:22.468241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:22.468241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:22.468241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:22.468241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:22.468241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:22.468241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:22.468241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:22.468241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:22.468241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:22.468241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:22.468241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:22.468241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:22.468241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:22.469404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:22.470502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:22.470502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:22.470502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:22.470502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:22.476596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 338/657] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:22.477545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915805c-3879-49d7-8b24-1edd170cac4a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:22.576794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499e40f-8db7-442d-917c-e79034b10d20 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:22.584061 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6915805c-3879-49d7-8b24-1edd170cac4a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:22.618631 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735dd248-b71a-4b9a-a4e3-5d68006b7d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:22.619590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735dd248-b71a-4b9a-a4e3-5d68006b7d14 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:22.620096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735dd248-b71a-4b9a-a4e3-5d68006b7d14 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:22.620475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735dd248-b71a-4b9a-a4e3-5d68006b7d14 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:22.621344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735dd248-b71a-4b9a-a4e3-5d68006b7d14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:22.621834 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735dd248-b71a-4b9a-a4e3-5d68006b7d14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:22.623285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735dd248-b71a-4b9a-a4e3-5d68006b7d14 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:22.625021 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f4f90-0d00-46c7-857a-269c7cb39621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:22.625695 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f4f90-0d00-46c7-857a-269c7cb39621 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:22.626474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f4f90-0d00-46c7-857a-269c7cb39621 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:22.626747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1f4f90-0d00-46c7-857a-269c7cb39621 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:22.637161 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 320/658] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:22.638091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:22.639469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735dd248-b71a-4b9a-a4e3-5d68006b7d14 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:22.700684 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:22.707275 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f4f90-0d00-46c7-857a-269c7cb39621 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:22.709006 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1f4f90-0d00-46c7-857a-269c7cb39621 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:22.758947 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735dd248-b71a-4b9a-a4e3-5d68006b7d14 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:22.798073 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e99ff4-9cd9-4ffd-9654-fdf4adb242bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:22.798737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e99ff4-9cd9-4ffd-9654-fdf4adb242bc None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:22.799019 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e99ff4-9cd9-4ffd-9654-fdf4adb242bc None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:22.799740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e99ff4-9cd9-4ffd-9654-fdf4adb242bc None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:22.800734 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e99ff4-9cd9-4ffd-9654-fdf4adb242bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:22.801083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e99ff4-9cd9-4ffd-9654-fdf4adb242bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:22.802920 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e99ff4-9cd9-4ffd-9654-fdf4adb242bc None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:22.812177 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 339/659] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:01:22.813084 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f4f90-0d00-46c7-857a-269c7cb39621 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:22.826404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 321/660] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:22.827754 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e55ae7-a1a4-41d3-a63d-fbfc12595a47 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:22.828048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e55ae7-a1a4-41d3-a63d-fbfc12595a47 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:22.828331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e55ae7-a1a4-41d3-a63d-fbfc12595a47 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:22.829727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 322/661] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:22.830565 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e55ae7-a1a4-41d3-a63d-fbfc12595a47 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:22.857082 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:22.918621 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1f4f90-0d00-46c7-857a-269c7cb39621 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:22.940935 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e55ae7-a1a4-41d3-a63d-fbfc12595a47 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:22.949848 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:22.952822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a52d2-c3f1-4b34-a95d-dc17dc9bfa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:22.953591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a52d2-c3f1-4b34-a95d-dc17dc9bfa82 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:22.953867 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a52d2-c3f1-4b34-a95d-dc17dc9bfa82 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:22.954140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a52d2-c3f1-4b34-a95d-dc17dc9bfa82 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:22.961247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a52d2-c3f1-4b34-a95d-dc17dc9bfa82 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:22.962666 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a52d2-c3f1-4b34-a95d-dc17dc9bfa82 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-6f2a52d2-c3f1-4b34-a95d-dc17dc9bfa82 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:22.977489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:22.978473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:22.979423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:22.979423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:22.979423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:22.979423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:22.979423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:22.979423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:22.979423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:22.979423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:22.979423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 14 10:01:22.979423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:22.979423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 340/662] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:22.979423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a52d2-c3f1-4b34-a95d-dc17dc9bfa82 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:22.985400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea36c9-7813-4f62-bb9e-572ce309f219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:22.986534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea36c9-7813-4f62-bb9e-572ce309f219 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:22.986807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea36c9-7813-4f62-bb9e-572ce309f219 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:22.987103 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea36c9-7813-4f62-bb9e-572ce309f219 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:22.997957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea36c9-7813-4f62-bb9e-572ce309f219 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:22.999378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ea36c9-7813-4f62-bb9e-572ce309f219 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:23.016499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-b8ea36c9-7813-4f62-bb9e-572ce309f219 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:23.016499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:23.016499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:23.016499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:23.016499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:23.016499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:23.016499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:23.016499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:23.016499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:23.016499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:23.016499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:23.016499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:23.016499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:23.016499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:23.016499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:23.017683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:23.018759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:23.018759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:23.018759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:23.018759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:23.018759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 323/663] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:23.019176 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2678a26-cff9-45db-8e39-86d5aee98ff7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:23.019456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2678a26-cff9-45db-8e39-86d5aee98ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:23.019725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2678a26-cff9-45db-8e39-86d5aee98ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:23.036124 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f2678a26-cff9-45db-8e39-86d5aee98ff7 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:23.037552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f2678a26-cff9-45db-8e39-86d5aee98ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:23.087375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:23.091373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a52d2-c3f1-4b34-a95d-dc17dc9bfa82 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:23.110606 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 324/664] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:23.112334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2678a26-cff9-45db-8e39-86d5aee98ff7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:23.129098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ecf9e-ae1e-49dc-88ee-819f283459aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:23.129942 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ecf9e-ae1e-49dc-88ee-819f283459aa None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:23.130307 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ecf9e-ae1e-49dc-88ee-819f283459aa None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:23.130582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ecf9e-ae1e-49dc-88ee-819f283459aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:23.138298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ecf9e-ae1e-49dc-88ee-819f283459aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:23.139764 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ecf9e-ae1e-49dc-88ee-819f283459aa None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:23.153957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-af7ecf9e-ae1e-49dc-88ee-819f283459aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:23.153957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:23.153957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:23.153957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:23.153957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:23.153957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:23.153957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:23.153957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:23.153957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:23.153957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:23.153957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:23.153957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:23.153957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:23.153957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:23.153957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:23.155073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:23.156171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:23.156171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:23.156171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:23.156171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:23.156171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:23.156171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:23.156171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:23.156171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:23.156171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 341/665] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:23.156171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ecf9e-ae1e-49dc-88ee-819f283459aa None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:23.231264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:23.243986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2678a26-cff9-45db-8e39-86d5aee98ff7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:23.270033 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ecf9e-ae1e-49dc-88ee-819f283459aa None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:23.280383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353d441-4536-436b-bfd0-49607559b0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:23.281114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353d441-4536-436b-bfd0-49607559b0ae None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:23.281395 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353d441-4536-436b-bfd0-49607559b0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:23.281663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6353d441-4536-436b-bfd0-49607559b0ae None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:23.282501 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6353d441-4536-436b-bfd0-49607559b0ae None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:23.282804 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6353d441-4536-436b-bfd0-49607559b0ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:23.284216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6353d441-4536-436b-bfd0-49607559b0ae None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:23.307021 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 325/666] 10.222.0.61 () {56 vars in 1139 bytes} [Thu Jul 14 10:01:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:23.307021 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353d441-4536-436b-bfd0-49607559b0ae None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:23.311600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282570fd-bd08-499c-bd3f-9fd3cd36d05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:23.312260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282570fd-bd08-499c-bd3f-9fd3cd36d05f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:23.312537 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282570fd-bd08-499c-bd3f-9fd3cd36d05f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:23.312807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282570fd-bd08-499c-bd3f-9fd3cd36d05f None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:23.313559 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282570fd-bd08-499c-bd3f-9fd3cd36d05f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:23.313863 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282570fd-bd08-499c-bd3f-9fd3cd36d05f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:23.315207 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282570fd-bd08-499c-bd3f-9fd3cd36d05f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:23.328604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 342/667] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:23.329702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282570fd-bd08-499c-bd3f-9fd3cd36d05f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:23.351049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:23.425463 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6353d441-4536-436b-bfd0-49607559b0ae None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:23.434700 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282570fd-bd08-499c-bd3f-9fd3cd36d05f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:23.462764 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3000ca2-f357-489b-a433-3604bc4984c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:23.463679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3000ca2-f357-489b-a433-3604bc4984c3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:23.463993 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3000ca2-f357-489b-a433-3604bc4984c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:23.464270 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3000ca2-f357-489b-a433-3604bc4984c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:23.469181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061e7e2-d85c-49e4-beef-2e2a058f1641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:23.469838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061e7e2-d85c-49e4-beef-2e2a058f1641 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:23.470115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061e7e2-d85c-49e4-beef-2e2a058f1641 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:23.470386 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a061e7e2-d85c-49e4-beef-2e2a058f1641 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:23.471170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061e7e2-d85c-49e4-beef-2e2a058f1641 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:23.471479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061e7e2-d85c-49e4-beef-2e2a058f1641 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:23.472898 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a061e7e2-d85c-49e4-beef-2e2a058f1641 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:23.475005 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3000ca2-f357-489b-a433-3604bc4984c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-a3000ca2-f357-489b-a433-3604bc4984c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:23.482716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:23.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:23.484629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:23.484629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:23.484629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:23.484629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:23.484629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:23.484629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:23.484629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:23.484629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:23.484629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:23.484629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:23.484629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:23.484629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:23.484629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 326/668] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:23.484629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3000ca2-f357-489b-a433-3604bc4984c3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:23.494731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:23.495696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 343/669] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:23.496618 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061e7e2-d85c-49e4-beef-2e2a058f1641 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:23.590983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:23.594274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3000ca2-f357-489b-a433-3604bc4984c3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:23.615059 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a061e7e2-d85c-49e4-beef-2e2a058f1641 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:23.627440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a19b98-2496-4780-8620-b0c6779c2eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:23.628092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a19b98-2496-4780-8620-b0c6779c2eba None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:23.628370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a19b98-2496-4780-8620-b0c6779c2eba None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:23.628637 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a19b98-2496-4780-8620-b0c6779c2eba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:23.653328 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec26ae-8d5b-4d52-b9a9-89c35e6fb7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:23.654124 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec26ae-8d5b-4d52-b9a9-89c35e6fb7cf None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:23.654405 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec26ae-8d5b-4d52-b9a9-89c35e6fb7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:23.654680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fec26ae-8d5b-4d52-b9a9-89c35e6fb7cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:23.661575 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec26ae-8d5b-4d52-b9a9-89c35e6fb7cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:23.663059 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fec26ae-8d5b-4d52-b9a9-89c35e6fb7cf None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:23.677523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-9fec26ae-8d5b-4d52-b9a9-89c35e6fb7cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:23.677523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:23.677523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:23.677523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:23.677523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:23.677523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:23.677523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:23.677523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:23.677523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:23.677523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:23.677523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:23.677523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:23.677523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:23.677523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:23.677523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:23.678707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:23.679837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:23.679837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:23.679837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:23.679837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:23.679837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:23.679837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:23.679837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:23.679837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:23.680552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 344/670] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:23.682199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec26ae-8d5b-4d52-b9a9-89c35e6fb7cf None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:23.703929 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a19b98-2496-4780-8620-b0c6779c2eba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:23.705650 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a19b98-2496-4780-8620-b0c6779c2eba None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:23.715281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:23.782549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fec26ae-8d5b-4d52-b9a9-89c35e6fb7cf None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:23.816881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 327/671] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:01:23.817748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a19b98-2496-4780-8620-b0c6779c2eba None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:23.826820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4b172-5b76-4435-b08d-b4c7dcce4dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:23.827489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4b172-5b76-4435-b08d-b4c7dcce4dfc None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:23.827769 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4b172-5b76-4435-b08d-b4c7dcce4dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:23.828041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4b172-5b76-4435-b08d-b4c7dcce4dfc None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:23.828788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4b172-5b76-4435-b08d-b4c7dcce4dfc None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:23.829125 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4b172-5b76-4435-b08d-b4c7dcce4dfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:23.830488 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4b172-5b76-4435-b08d-b4c7dcce4dfc None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:23.836211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:23.840042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 345/672] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:23.841144 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4b172-5b76-4435-b08d-b4c7dcce4dfc None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:23.935257 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a19b98-2496-4780-8620-b0c6779c2eba None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:23.943298 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:23.955135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4b172-5b76-4435-b08d-b4c7dcce4dfc None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:23.973747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4e412-c99b-410f-b3d4-d136453ac89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:23.974533 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4e412-c99b-410f-b3d4-d136453ac89a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:23.974805 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4e412-c99b-410f-b3d4-d136453ac89a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:23.975284 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be4e412-c99b-410f-b3d4-d136453ac89a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:23.976029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4e412-c99b-410f-b3d4-d136453ac89a None admin] RBAC: Adding query filter params (role.id=ba4088a289ea4b9a9aa9d9cd7a27e731, scope.project.id=57662256880f43018039715a755c087c, user.id=52e3fbd0400a49a69932ce426218f4a6) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:23.976341 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4e412-c99b-410f-b3d4-d136453ac89a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:23.978052 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be4e412-c99b-410f-b3d4-d136453ac89a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:23.993335 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 328/673] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:23 2022] GET /identity/v3/role_assignments?user.id=52e3fbd0400a49a69932ce426218f4a6&scope.project.id=57662256880f43018039715a755c087c&role.id=ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:23.993335 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4e412-c99b-410f-b3d4-d136453ac89a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:23.995162 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a7069-860e-4f5f-b02e-271afe4b12e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:23.996144 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a7069-860e-4f5f-b02e-271afe4b12e3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:23.996476 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a7069-860e-4f5f-b02e-271afe4b12e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:23.996756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38a7069-860e-4f5f-b02e-271afe4b12e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:23.997616 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a7069-860e-4f5f-b02e-271afe4b12e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:23.997930 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a7069-860e-4f5f-b02e-271afe4b12e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:23.999423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38a7069-860e-4f5f-b02e-271afe4b12e3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:24.013840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 346/674] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:24.019614 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a7069-860e-4f5f-b02e-271afe4b12e3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:24.090433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:24.119585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be4e412-c99b-410f-b3d4-d136453ac89a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:24.137758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38a7069-860e-4f5f-b02e-271afe4b12e3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:24.160653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f2831-1ab9-4481-9c21-4ebcddcd0154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:24.161486 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f2831-1ab9-4481-9c21-4ebcddcd0154 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:24.161768 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f2831-1ab9-4481-9c21-4ebcddcd0154 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:24.162884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f2831-1ab9-4481-9c21-4ebcddcd0154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:24.175533 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f2831-1ab9-4481-9c21-4ebcddcd0154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:24.176996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64f2831-1ab9-4481-9c21-4ebcddcd0154 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:24.178524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe068b-05f2-4f5d-a01b-4d0f2730b10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:24.179331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe068b-05f2-4f5d-a01b-4d0f2730b10b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:24.179615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe068b-05f2-4f5d-a01b-4d0f2730b10b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:24.179885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe068b-05f2-4f5d-a01b-4d0f2730b10b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:24.191067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe068b-05f2-4f5d-a01b-4d0f2730b10b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:24.192508 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebe068b-05f2-4f5d-a01b-4d0f2730b10b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:24.197169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-b64f2831-1ab9-4481-9c21-4ebcddcd0154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:24.197169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:24.197169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:24.197169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:24.197169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:24.197169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:24.197169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:24.197169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:24.197169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:24.197169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:24.197169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:24.197169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:24.197169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:24.197169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:24.197169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:24.198060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:24.198989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:24.198989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:24.198989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:24.198989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:24.198989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 329/675] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:24.199269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f2831-1ab9-4481-9c21-4ebcddcd0154 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:24.211532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-cebe068b-05f2-4f5d-a01b-4d0f2730b10b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:24.211532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:24.211532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:24.211532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:24.211532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:24.211532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:24.211532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:24.211532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:24.211532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:24.211532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:24.211532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:24.211532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:24.211532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:24.211532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:24.211532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:24.212455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:24.213766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:24.213766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:24.213766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:24.213766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:24.213766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 347/676] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:24.214053 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf03ff2-c05b-4fde-98e2-b276ef89cd00 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:24.214589 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf03ff2-c05b-4fde-98e2-b276ef89cd00 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:24.214589 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf03ff2-c05b-4fde-98e2-b276ef89cd00 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:24.215900 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 348/677] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:24.216441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf03ff2-c05b-4fde-98e2-b276ef89cd00 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:24.239461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:24.325545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64f2831-1ab9-4481-9c21-4ebcddcd0154 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:24.330671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf03ff2-c05b-4fde-98e2-b276ef89cd00 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:24.344315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:24.366149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67c271-4412-4ae1-8569-7ffb9db134ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:24.366770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67c271-4412-4ae1-8569-7ffb9db134ec None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:24.367028 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67c271-4412-4ae1-8569-7ffb9db134ec None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:24.367348 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67c271-4412-4ae1-8569-7ffb9db134ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:24.368133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67c271-4412-4ae1-8569-7ffb9db134ec None admin] RBAC: Adding query filter params (role.id=ba4088a289ea4b9a9aa9d9cd7a27e731, scope.project.id=57662256880f43018039715a755c087c, user.id=e3b5660cfd344dcd8abe14c88df686e8) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:24.368444 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67c271-4412-4ae1-8569-7ffb9db134ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:24.369044 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b930d-5ff1-4299-8538-4453f9d727bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:24.369772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b930d-5ff1-4299-8538-4453f9d727bd None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:24.369883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67c271-4412-4ae1-8569-7ffb9db134ec None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:24.370167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b930d-5ff1-4299-8538-4453f9d727bd None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:24.370511 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b930d-5ff1-4299-8538-4453f9d727bd None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:24.371306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b930d-5ff1-4299-8538-4453f9d727bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:24.371701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b930d-5ff1-4299-8538-4453f9d727bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:24.373132 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b930d-5ff1-4299-8538-4453f9d727bd None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:24.376749 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 330/678] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:24 2022] GET /identity/v3/role_assignments?user.id=e3b5660cfd344dcd8abe14c88df686e8&scope.project.id=57662256880f43018039715a755c087c&role.id=ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:24.377706 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67c271-4412-4ae1-8569-7ffb9db134ec None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:24.385815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 349/679] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:24.387210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200e7c4-d53a-4767-97e2-bcefbaabf545 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:24.387504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200e7c4-d53a-4767-97e2-bcefbaabf545 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:24.387780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2200e7c4-d53a-4767-97e2-bcefbaabf545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:24.407224 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2200e7c4-d53a-4767-97e2-bcefbaabf545 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:24.408481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2200e7c4-d53a-4767-97e2-bcefbaabf545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:24.500644 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 350/680] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:24.501624 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200e7c4-d53a-4767-97e2-bcefbaabf545 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:24.507302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:24.525423 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67c271-4412-4ae1-8569-7ffb9db134ec None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:24.564795 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81afc45-b667-45d7-88b3-b7c0a4f05ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:24.565558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81afc45-b667-45d7-88b3-b7c0a4f05ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:24.565856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81afc45-b667-45d7-88b3-b7c0a4f05ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:24.566127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81afc45-b667-45d7-88b3-b7c0a4f05ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:24.566869 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81afc45-b667-45d7-88b3-b7c0a4f05ce3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:24.567181 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81afc45-b667-45d7-88b3-b7c0a4f05ce3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:24.568585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81afc45-b667-45d7-88b3-b7c0a4f05ce3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:24.582080 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 331/681] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:24.583040 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81afc45-b667-45d7-88b3-b7c0a4f05ce3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:24.639188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2200e7c4-d53a-4767-97e2-bcefbaabf545 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:24.641769 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:24.683570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c5cda-ba16-4a14-9390-ef72ca3b8cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:24.684515 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c5cda-ba16-4a14-9390-ef72ca3b8cec None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:24.684898 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c5cda-ba16-4a14-9390-ef72ca3b8cec None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:24.685307 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922c5cda-ba16-4a14-9390-ef72ca3b8cec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:24.693920 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c5cda-ba16-4a14-9390-ef72ca3b8cec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:24.696455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922c5cda-ba16-4a14-9390-ef72ca3b8cec None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:24.706242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81afc45-b667-45d7-88b3-b7c0a4f05ce3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:24.711897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-922c5cda-ba16-4a14-9390-ef72ca3b8cec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:24.711897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:24.711897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:24.711897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:24.711897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:24.711897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:24.711897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:24.711897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:24.711897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:24.711897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:24.711897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:24.711897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:24.711897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:24.711897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:24.711897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:24.712971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:24.714088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:24.714088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:24.714088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:24.714088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:24.714088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:24.714088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:24.714088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:24.714088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:24.715883 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 351/682] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:24.717141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c5cda-ba16-4a14-9390-ef72ca3b8cec None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:24.748003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8a844-8a82-49ad-a488-ea0d974dacf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:24.748843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8a844-8a82-49ad-a488-ea0d974dacf7 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:24.749196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8a844-8a82-49ad-a488-ea0d974dacf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:24.749471 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8a844-8a82-49ad-a488-ea0d974dacf7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:24.761475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8a844-8a82-49ad-a488-ea0d974dacf7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-15a8a844-8a82-49ad-a488-ea0d974dacf7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:24.769660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:24.770807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:24.772512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:24.772512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:24.772512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:24.772512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:24.772512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:24.772512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:24.772512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:24.772512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:24.772512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:24.772512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:24.772512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:01:24.772512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:24.772512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 332/683] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:01:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:24.772512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8a844-8a82-49ad-a488-ea0d974dacf7 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:24.787102 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:24.867693 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922c5cda-ba16-4a14-9390-ef72ca3b8cec None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:24.885033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8a844-8a82-49ad-a488-ea0d974dacf7 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:24.917607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b074a34-ae77-4e8c-920c-e13dc947b623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:24.919290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b074a34-ae77-4e8c-920c-e13dc947b623 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:24.919991 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b074a34-ae77-4e8c-920c-e13dc947b623 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:24.920543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b074a34-ae77-4e8c-920c-e13dc947b623 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:24.923731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f41030-00cc-41fe-b82f-b47637ca5f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:24.924491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f41030-00cc-41fe-b82f-b47637ca5f8c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:24.924656 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f41030-00cc-41fe-b82f-b47637ca5f8c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:24.924946 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f41030-00cc-41fe-b82f-b47637ca5f8c None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:24.925743 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f41030-00cc-41fe-b82f-b47637ca5f8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:24.926051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f41030-00cc-41fe-b82f-b47637ca5f8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:24.927484 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f41030-00cc-41fe-b82f-b47637ca5f8c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:24.931144 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b074a34-ae77-4e8c-920c-e13dc947b623 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:24.933095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b074a34-ae77-4e8c-920c-e13dc947b623 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:24.941427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 333/684] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:24.942265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f41030-00cc-41fe-b82f-b47637ca5f8c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-8b074a34-ae77-4e8c-920c-e13dc947b623 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:24.957281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:24.958433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:24.960571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:24.960571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:24.960571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:24.960571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:24.960571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:24.960571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:24.960571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:24.960571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:24.960571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:01:24.960571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:24.969532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 352/685] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:01:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:24.971199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b074a34-ae77-4e8c-920c-e13dc947b623 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:24.999206 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:25.024000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:25.062194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f41030-00cc-41fe-b82f-b47637ca5f8c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:25.100200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b074a34-ae77-4e8c-920c-e13dc947b623 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:25.105959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f76a0d-21f7-4318-9c31-444438d43589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:25.106835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f76a0d-21f7-4318-9c31-444438d43589 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:25.107775 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f76a0d-21f7-4318-9c31-444438d43589 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:25.108206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f76a0d-21f7-4318-9c31-444438d43589 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:25.109599 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f76a0d-21f7-4318-9c31-444438d43589 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:25.110081 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f76a0d-21f7-4318-9c31-444438d43589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:25.112357 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f76a0d-21f7-4318-9c31-444438d43589 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:25.125180 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 334/686] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:01:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:25.126299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f76a0d-21f7-4318-9c31-444438d43589 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:25.141006 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bc491-8b27-43f0-9a29-daedf28d0304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:25.141685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bc491-8b27-43f0-9a29-daedf28d0304 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:25.141961 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bc491-8b27-43f0-9a29-daedf28d0304 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:25.142231 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98bc491-8b27-43f0-9a29-daedf28d0304 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:25.143090 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bc491-8b27-43f0-9a29-daedf28d0304 None admin] RBAC: Adding query filter params (role.id=ba4088a289ea4b9a9aa9d9cd7a27e731, scope.project.id=57662256880f43018039715a755c087c, user.id=cd6d19c4406b4b228065b98a9a074b3a) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:25.143649 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bc491-8b27-43f0-9a29-daedf28d0304 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:25.144847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98bc491-8b27-43f0-9a29-daedf28d0304 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:25.152308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 353/687] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:24 2022] GET /identity/v3/role_assignments?user.id=cd6d19c4406b4b228065b98a9a074b3a&scope.project.id=57662256880f43018039715a755c087c&role.id=ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:25.153187 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bc491-8b27-43f0-9a29-daedf28d0304 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:25.260109 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f76a0d-21f7-4318-9c31-444438d43589 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:25.292669 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:25.303827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:25.306170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0675a-0770-4d5a-afd0-e2f412e4f9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:25.307032 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0675a-0770-4d5a-afd0-e2f412e4f9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:25.307722 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0675a-0770-4d5a-afd0-e2f412e4f9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:25.308162 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0675a-0770-4d5a-afd0-e2f412e4f9b3 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:25.309263 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0675a-0770-4d5a-afd0-e2f412e4f9b3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:25.309762 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0675a-0770-4d5a-afd0-e2f412e4f9b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:25.311525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0675a-0770-4d5a-afd0-e2f412e4f9b3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:25.319366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98bc491-8b27-43f0-9a29-daedf28d0304 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:25.339358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 335/688] 10.222.0.61 () {56 vars in 1135 bytes} [Thu Jul 14 10:01:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:25.340974 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0675a-0770-4d5a-afd0-e2f412e4f9b3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:25.356345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be770f-28bb-4a1e-b46c-7c0005da1bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:25.357250 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be770f-28bb-4a1e-b46c-7c0005da1bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:25.357599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be770f-28bb-4a1e-b46c-7c0005da1bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:25.357941 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be770f-28bb-4a1e-b46c-7c0005da1bc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:25.369128 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be770f-28bb-4a1e-b46c-7c0005da1bc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:25.370647 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be770f-28bb-4a1e-b46c-7c0005da1bc1 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:25.388037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-47be770f-28bb-4a1e-b46c-7c0005da1bc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:25.388037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:25.388037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:25.388037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:25.388037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:25.388037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:25.388037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:25.388037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:25.388037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:25.388037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:25.388037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:25.388037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:25.388037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:25.388037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:25.388037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:25.389182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:25.390307 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:25.390307 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:25.390307 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:25.390307 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:25.391388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 354/689] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:25.397212 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be770f-28bb-4a1e-b46c-7c0005da1bc1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:25.456997 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0675a-0770-4d5a-afd0-e2f412e4f9b3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:25.495701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c55a87-882e-4b19-a37c-cc8cf0049f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:25.496590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55a87-882e-4b19-a37c-cc8cf0049f37 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:25.497219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55a87-882e-4b19-a37c-cc8cf0049f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:25.497659 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c55a87-882e-4b19-a37c-cc8cf0049f37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:25.499446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be770f-28bb-4a1e-b46c-7c0005da1bc1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:25.534054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17567b8e-21c5-408f-983a-2b642f415c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:25.534785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17567b8e-21c5-408f-983a-2b642f415c7d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:25.535159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17567b8e-21c5-408f-983a-2b642f415c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:25.535531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17567b8e-21c5-408f-983a-2b642f415c7d None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:25.536568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17567b8e-21c5-408f-983a-2b642f415c7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:25.536744 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17567b8e-21c5-408f-983a-2b642f415c7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:25.538861 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17567b8e-21c5-408f-983a-2b642f415c7d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:25.546171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:25.551219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 355/690] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:25.556769 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17567b8e-21c5-408f-983a-2b642f415c7d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:25.559963 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:25.578575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c55a87-882e-4b19-a37c-cc8cf0049f37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:25.580697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c55a87-882e-4b19-a37c-cc8cf0049f37 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:25.675728 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17567b8e-21c5-408f-983a-2b642f415c7d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:25.714185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2ad51-16e2-4a7b-ae59-32d189aa4720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:25.715299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2ad51-16e2-4a7b-ae59-32d189aa4720 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:25.715683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2ad51-16e2-4a7b-ae59-32d189aa4720 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:25.716056 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2ad51-16e2-4a7b-ae59-32d189aa4720 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:25.724635 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2ad51-16e2-4a7b-ae59-32d189aa4720 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:25.726498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2ad51-16e2-4a7b-ae59-32d189aa4720 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:25.734983 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 336/691] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-8ea2ad51-16e2-4a7b-ae59-32d189aa4720 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:25.742640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:25.743882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:25.745115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:25.745115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:25.745115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:25.745115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:25.745115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:25.745115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:25.745115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:25.745115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:25.745115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:01:25.745115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:25.746797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c55a87-882e-4b19-a37c-cc8cf0049f37 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:25.746917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 356/692] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Jul 14 10:01:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:25.753058 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2ad51-16e2-4a7b-ae59-32d189aa4720 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:25.829416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:25.892893 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2ad51-16e2-4a7b-ae59-32d189aa4720 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:25.900713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:25.917950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c55a87-882e-4b19-a37c-cc8cf0049f37 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:25.955623 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f4b32-dfe6-40a0-8167-31fab49e8b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:25.956686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f4b32-dfe6-40a0-8167-31fab49e8b70 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:25.957157 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f4b32-dfe6-40a0-8167-31fab49e8b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:25.957574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88f4b32-dfe6-40a0-8167-31fab49e8b70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:25.961617 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32d083-5efc-455c-a40d-759b9c7723b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:25.962298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32d083-5efc-455c-a40d-759b9c7723b3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:25.962578 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32d083-5efc-455c-a40d-759b9c7723b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:25.962848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d32d083-5efc-455c-a40d-759b9c7723b3 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:25.963696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32d083-5efc-455c-a40d-759b9c7723b3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:25.963998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32d083-5efc-455c-a40d-759b9c7723b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:25.965461 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d32d083-5efc-455c-a40d-759b9c7723b3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:25.970411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f4b32-dfe6-40a0-8167-31fab49e8b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:25.972004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88f4b32-dfe6-40a0-8167-31fab49e8b70 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:25.990837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-b88f4b32-dfe6-40a0-8167-31fab49e8b70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:25.990837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:25.990837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:25.990837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:25.990837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:25.990837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:25.990837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:25.990837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:25.990837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:25.990837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:25.990837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:25.990837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:25.990837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:25.990837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:25.990837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:25.992077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:25.993192 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:25.993192 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:25.993192 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:25.993192 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:25.999350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 357/694] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Jul 14 10:01:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:25.999350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32d083-5efc-455c-a40d-759b9c7723b3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:25.999746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 337/694] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:26.001073 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f4b32-dfe6-40a0-8167-31fab49e8b70 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:26.108738 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88f4b32-dfe6-40a0-8167-31fab49e8b70 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:26.146056 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e19647-f71f-4cb3-a855-48bd75c2c564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:26.147083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e19647-f71f-4cb3-a855-48bd75c2c564 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:26.147561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e19647-f71f-4cb3-a855-48bd75c2c564 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:26.147998 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e19647-f71f-4cb3-a855-48bd75c2c564 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:26.148874 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e19647-f71f-4cb3-a855-48bd75c2c564 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:26.149346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e19647-f71f-4cb3-a855-48bd75c2c564 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:26.150878 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e19647-f71f-4cb3-a855-48bd75c2c564 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:26.158580 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:26.169327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 338/695] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:26.171195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e19647-f71f-4cb3-a855-48bd75c2c564 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:26.272345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d32d083-5efc-455c-a40d-759b9c7723b3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:26.282380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e19647-f71f-4cb3-a855-48bd75c2c564 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:26.293584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:26.308212 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec4751-13d6-4263-9d33-c8729c1865b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:26.308875 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec4751-13d6-4263-9d33-c8729c1865b8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:26.309193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec4751-13d6-4263-9d33-c8729c1865b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:26.309474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec4751-13d6-4263-9d33-c8729c1865b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:26.327537 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a939b1b1-0d19-4a58-99c4-3f90b33d1458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:26.328650 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939b1b1-0d19-4a58-99c4-3f90b33d1458 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:26.328948 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939b1b1-0d19-4a58-99c4-3f90b33d1458 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:26.329252 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a939b1b1-0d19-4a58-99c4-3f90b33d1458 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:26.338857 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939b1b1-0d19-4a58-99c4-3f90b33d1458 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:26.340319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a939b1b1-0d19-4a58-99c4-3f90b33d1458 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:26.354905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-a939b1b1-0d19-4a58-99c4-3f90b33d1458 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:26.354905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:26.354905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:26.354905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:26.354905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:26.354905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:26.354905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:26.354905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:26.354905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:26.354905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:26.354905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:26.354905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:26.354905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:26.354905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:26.354905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:26.355833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:26.356747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:26.356747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:26.356747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:26.356747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:26.356747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:26.356747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:26.356747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:26.356747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:26.356747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 339/696] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:26.358675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f1fd0-c8bf-4805-97b3-f0f4e088b28b None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:26.358675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f1fd0-c8bf-4805-97b3-f0f4e088b28b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:26.358675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9f1fd0-c8bf-4805-97b3-f0f4e088b28b None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:26.363700 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 340/697] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:26.365687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39908bac-0e85-4b3a-8b49-db04d6e46d67 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:26.365977 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39908bac-0e85-4b3a-8b49-db04d6e46d67 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:26.366257 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39908bac-0e85-4b3a-8b49-db04d6e46d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:26.383838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-39908bac-0e85-4b3a-8b49-db04d6e46d67 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:26.385068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-39908bac-0e85-4b3a-8b49-db04d6e46d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:26.395406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec4751-13d6-4263-9d33-c8729c1865b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:26.397849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec4751-13d6-4263-9d33-c8729c1865b8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:26.442711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:26.464134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 341/698] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:26.465042 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39908bac-0e85-4b3a-8b49-db04d6e46d67 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:26.526078 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 358/699] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:01:26.526996 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec4751-13d6-4263-9d33-c8729c1865b8 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:26.544905 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:26.601229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39908bac-0e85-4b3a-8b49-db04d6e46d67 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:26.647273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95cd4b-ca58-4e76-8668-027d29d29720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:26.647606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec4751-13d6-4263-9d33-c8729c1865b8 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:26.647925 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95cd4b-ca58-4e76-8668-027d29d29720 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:26.648174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95cd4b-ca58-4e76-8668-027d29d29720 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:26.648452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95cd4b-ca58-4e76-8668-027d29d29720 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:26.650427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95cd4b-ca58-4e76-8668-027d29d29720 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:26.650765 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95cd4b-ca58-4e76-8668-027d29d29720 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:26.652298 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95cd4b-ca58-4e76-8668-027d29d29720 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:26.673024 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 342/700] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:26.673895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95cd4b-ca58-4e76-8668-027d29d29720 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:26.689711 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276addbe-299a-4349-ab3e-d96a5034d67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:26.690573 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276addbe-299a-4349-ab3e-d96a5034d67c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:26.690887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276addbe-299a-4349-ab3e-d96a5034d67c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:26.691177 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276addbe-299a-4349-ab3e-d96a5034d67c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:26.691850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276addbe-299a-4349-ab3e-d96a5034d67c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:26.693447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276addbe-299a-4349-ab3e-d96a5034d67c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-276addbe-299a-4349-ab3e-d96a5034d67c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:26.712029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:26.714374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:26.716455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 359/701] 10.222.0.61 () {56 vars in 1129 bytes} [Thu Jul 14 10:01:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:26.718676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276addbe-299a-4349-ab3e-d96a5034d67c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:26.739129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:26.804006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95cd4b-ca58-4e76-8668-027d29d29720 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:26.811268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:26.834440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276addbe-299a-4349-ab3e-d96a5034d67c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:26.842624 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35c898-ebc5-44d8-b806-ff7f45a2a5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:26.843480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35c898-ebc5-44d8-b806-ff7f45a2a5ec None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:26.843795 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35c898-ebc5-44d8-b806-ff7f45a2a5ec None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:26.844071 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35c898-ebc5-44d8-b806-ff7f45a2a5ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:26.855683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35c898-ebc5-44d8-b806-ff7f45a2a5ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:26.857149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af35c898-ebc5-44d8-b806-ff7f45a2a5ec None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:26.872096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a7f28b-a373-48e5-93c7-936b8d3d7134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:26.872751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7f28b-a373-48e5-93c7-936b8d3d7134 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:26.873063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7f28b-a373-48e5-93c7-936b8d3d7134 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:26.873335 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a7f28b-a373-48e5-93c7-936b8d3d7134 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:26.874164 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7f28b-a373-48e5-93c7-936b8d3d7134 None admin] RBAC: Adding query filter params (role.id=ba4088a289ea4b9a9aa9d9cd7a27e731, scope.project.id=57662256880f43018039715a755c087c, user.id=d5fa502e8b014c719fcece0a130fdd7e) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:26.874472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7f28b-a373-48e5-93c7-936b8d3d7134 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:26.875882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a7f28b-a373-48e5-93c7-936b8d3d7134 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:26.876187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-af35c898-ebc5-44d8-b806-ff7f45a2a5ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:26.876187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:26.876187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:26.876187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:26.876187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:26.876187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:26.876187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:26.876187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:26.876187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:26.876187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:26.876187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:26.876187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:26.876187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:26.876187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:26.876187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:26.877089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:26.878020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:26.878020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:26.878020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:26.878020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:26.882160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 343/702] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:26.883042 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35c898-ebc5-44d8-b806-ff7f45a2a5ec None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:26.889540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 360/703] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:26 2022] GET /identity/v3/role_assignments?user.id=d5fa502e8b014c719fcece0a130fdd7e&scope.project.id=57662256880f43018039715a755c087c&role.id=ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:26.889540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136226fd-e65f-4b48-9353-60470b21f924 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:26.889540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136226fd-e65f-4b48-9353-60470b21f924 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:26.889540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136226fd-e65f-4b48-9353-60470b21f924 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:26.889540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 361/704] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:26.889540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136226fd-e65f-4b48-9353-60470b21f924 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:27.003321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:27.007756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35c898-ebc5-44d8-b806-ff7f45a2a5ec None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:27.041922 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136226fd-e65f-4b48-9353-60470b21f924 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:27.051482 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a726eea-f49d-403a-b6f3-22cc47a75e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:27.052576 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a726eea-f49d-403a-b6f3-22cc47a75e2a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:27.053109 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a726eea-f49d-403a-b6f3-22cc47a75e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:27.053582 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a726eea-f49d-403a-b6f3-22cc47a75e2a None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:27.054497 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a726eea-f49d-403a-b6f3-22cc47a75e2a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:27.055007 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a726eea-f49d-403a-b6f3-22cc47a75e2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:27.056635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a726eea-f49d-403a-b6f3-22cc47a75e2a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:27.062869 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 344/705] 10.222.0.61 () {56 vars in 1124 bytes} [Thu Jul 14 10:01:26 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:27.064466 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78407d9-8543-43dc-9ee5-558a88e6f41b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:27.064966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78407d9-8543-43dc-9ee5-558a88e6f41b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:27.065423 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78407d9-8543-43dc-9ee5-558a88e6f41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:27.086719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a78407d9-8543-43dc-9ee5-558a88e6f41b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:27.087883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a78407d9-8543-43dc-9ee5-558a88e6f41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:27.088362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cceec5-da56-46a0-a03e-f2dccb30f00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:27.089191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cceec5-da56-46a0-a03e-f2dccb30f00e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:27.089538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cceec5-da56-46a0-a03e-f2dccb30f00e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:27.089958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cceec5-da56-46a0-a03e-f2dccb30f00e None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:27.090877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cceec5-da56-46a0-a03e-f2dccb30f00e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:27.091278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cceec5-da56-46a0-a03e-f2dccb30f00e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:27.092716 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cceec5-da56-46a0-a03e-f2dccb30f00e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:27.106822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 362/706] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:27.107568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cceec5-da56-46a0-a03e-f2dccb30f00e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:27.111843 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:27.166018 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 345/707] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:27.167126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78407d9-8543-43dc-9ee5-558a88e6f41b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:27.227771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cceec5-da56-46a0-a03e-f2dccb30f00e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:27.274002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501eb97-b37c-4559-bd5c-a297f0d7e439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:27.274604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501eb97-b37c-4559-bd5c-a297f0d7e439 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:27.274891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501eb97-b37c-4559-bd5c-a297f0d7e439 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:27.275165 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3501eb97-b37c-4559-bd5c-a297f0d7e439 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:27.275887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501eb97-b37c-4559-bd5c-a297f0d7e439 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:27.276216 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501eb97-b37c-4559-bd5c-a297f0d7e439 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:27.277625 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3501eb97-b37c-4559-bd5c-a297f0d7e439 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:27.284189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 363/708] 10.222.0.61 () {56 vars in 1124 bytes} [Thu Jul 14 10:01:27 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:27.285143 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501eb97-b37c-4559-bd5c-a297f0d7e439 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:27.290972 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:27.301688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78407d9-8543-43dc-9ee5-558a88e6f41b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:27.387338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff00e2-b288-4e3a-a739-dcd39c039c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:27.388206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff00e2-b288-4e3a-a739-dcd39c039c62 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:27.388518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff00e2-b288-4e3a-a739-dcd39c039c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:27.388803 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff00e2-b288-4e3a-a739-dcd39c039c62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:27.410327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff00e2-b288-4e3a-a739-dcd39c039c62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:27.410983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:27.412007 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ff00e2-b288-4e3a-a739-dcd39c039c62 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:27.441846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-b1ff00e2-b288-4e3a-a739-dcd39c039c62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:27.441846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:27.441846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:27.441846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:27.441846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:27.441846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:27.441846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:27.441846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:27.441846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:27.441846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:27.441846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:27.441846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:27.441846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:27.441846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:27.441846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:27.443343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:27.444232 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:27.444232 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:27.444232 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:27.444232 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:27.444232 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:27.444232 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:27.444232 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:27.444232 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:27.446906 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 346/709] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:27.458495 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff00e2-b288-4e3a-a739-dcd39c039c62 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:27.472621 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3501eb97-b37c-4559-bd5c-a297f0d7e439 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:27.520655 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc355eb-baa4-4212-80d7-8eb7e1bd63d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:27.522212 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc355eb-baa4-4212-80d7-8eb7e1bd63d8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:27.522499 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc355eb-baa4-4212-80d7-8eb7e1bd63d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:27.522824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc355eb-baa4-4212-80d7-8eb7e1bd63d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:27.535128 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc355eb-baa4-4212-80d7-8eb7e1bd63d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:27.536654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc355eb-baa4-4212-80d7-8eb7e1bd63d8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:27.555495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-abc355eb-baa4-4212-80d7-8eb7e1bd63d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:27.555495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:27.555495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:27.555495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:27.555495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:27.555495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:27.555495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:27.555495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:27.555495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:27.555495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:27.555495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:27.555495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:27.555495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:27.555495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:27.555495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:27.557190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:27.558088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:27.558088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:27.558088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:27.558088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:27.558088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 364/710] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:27.569900 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc355eb-baa4-4212-80d7-8eb7e1bd63d8 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:27.592724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff00e2-b288-4e3a-a739-dcd39c039c62 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:27.657644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f28afe-8fd4-4544-9edc-80114c8c3415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:27.658295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f28afe-8fd4-4544-9edc-80114c8c3415 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:27.658582 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f28afe-8fd4-4544-9edc-80114c8c3415 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:27.658852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f28afe-8fd4-4544-9edc-80114c8c3415 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:27.659653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f28afe-8fd4-4544-9edc-80114c8c3415 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:27.659943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f28afe-8fd4-4544-9edc-80114c8c3415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:27.661405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f28afe-8fd4-4544-9edc-80114c8c3415 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:27.679036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 347/711] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:27.684830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f28afe-8fd4-4544-9edc-80114c8c3415 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:27.695472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc355eb-baa4-4212-80d7-8eb7e1bd63d8 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:27.714272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:27.717123 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:27.734167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9838197-7b50-407c-90f1-7a9b1f66ee9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:27.735073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9838197-7b50-407c-90f1-7a9b1f66ee9c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:27.735492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9838197-7b50-407c-90f1-7a9b1f66ee9c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:27.736004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9838197-7b50-407c-90f1-7a9b1f66ee9c None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:27.736885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9838197-7b50-407c-90f1-7a9b1f66ee9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:27.737488 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9838197-7b50-407c-90f1-7a9b1f66ee9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:27.739350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9838197-7b50-407c-90f1-7a9b1f66ee9c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:27.754775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 365/712] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:27.763061 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9838197-7b50-407c-90f1-7a9b1f66ee9c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:27.884384 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f28afe-8fd4-4544-9edc-80114c8c3415 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:27.924637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9838197-7b50-407c-90f1-7a9b1f66ee9c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:27.929373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0499a2-fa56-4032-b383-c24ab410e7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:27.930132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0499a2-fa56-4032-b383-c24ab410e7ce None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:27.930422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0499a2-fa56-4032-b383-c24ab410e7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:27.930699 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0499a2-fa56-4032-b383-c24ab410e7ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:27.931791 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0499a2-fa56-4032-b383-c24ab410e7ce None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=57662256880f43018039715a755c087c, user.id=e9f4efa3e2f343c2ac858a277601a7df) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:27.932108 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0499a2-fa56-4032-b383-c24ab410e7ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:27.933572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0499a2-fa56-4032-b383-c24ab410e7ce None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:27.945034 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 348/713] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:27 2022] GET /identity/v3/role_assignments?user.id=e9f4efa3e2f343c2ac858a277601a7df&scope.project.id=57662256880f43018039715a755c087c&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:27.946885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831d5d1-d116-4210-a149-73631656fe06 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:27.947377 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831d5d1-d116-4210-a149-73631656fe06 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:27.947846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831d5d1-d116-4210-a149-73631656fe06 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:27.952527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 349/714] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:27.958548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091156f-91e2-4e14-aed7-b8b22f0f5f3f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:27.958548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091156f-91e2-4e14-aed7-b8b22f0f5f3f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:27.958548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7091156f-91e2-4e14-aed7-b8b22f0f5f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:27.982222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7091156f-91e2-4e14-aed7-b8b22f0f5f3f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:27.983451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7091156f-91e2-4e14-aed7-b8b22f0f5f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:28.005069 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde95f6e-c6f0-4e28-a0e5-34c9c5339345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:28.006011 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde95f6e-c6f0-4e28-a0e5-34c9c5339345 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:28.006293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde95f6e-c6f0-4e28-a0e5-34c9c5339345 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:28.006569 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde95f6e-c6f0-4e28-a0e5-34c9c5339345 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:28.007303 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde95f6e-c6f0-4e28-a0e5-34c9c5339345 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:28.008793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde95f6e-c6f0-4e28-a0e5-34c9c5339345 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:28.047566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 366/715] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:01:27 2022] POST /identity/v3/users => generated 254 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:01:28.123545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 350/716] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:01:28.151663 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde95f6e-c6f0-4e28-a0e5-34c9c5339345 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:28.196835 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:28.294394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde95f6e-c6f0-4e28-a0e5-34c9c5339345 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:28.334614 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881e174-ad9c-47ee-9b83-653417bc16a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:28.335429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e174-ad9c-47ee-9b83-653417bc16a7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:28.335707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e174-ad9c-47ee-9b83-653417bc16a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:28.335981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8881e174-ad9c-47ee-9b83-653417bc16a7 None admin] PATH_INFO: `/v3/services/network` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:28.336648 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881e174-ad9c-47ee-9b83-653417bc16a7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:28.338043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8881e174-ad9c-47ee-9b83-653417bc16a7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-8881e174-ad9c-47ee-9b83-653417bc16a7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:28.352041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:28.353169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:28.354034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:28.354034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:28.354034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:28.354034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:28.354034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:28.354034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:28.354034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:28.354034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:28.354034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:28.354034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:28.354034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:28.354034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:28.354034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:01:28.354034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:28.354034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 367/717] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:28 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:28.359485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091156f-91e2-4e14-aed7-b8b22f0f5f3f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:28.376255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:28.477718 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7091156f-91e2-4e14-aed7-b8b22f0f5f3f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:28.518735 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca4444-d9be-4061-ad16-40f4317e4e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:28.519787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca4444-d9be-4061-ad16-40f4317e4e9a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:28.520263 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca4444-d9be-4061-ad16-40f4317e4e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:28.521287 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca4444-d9be-4061-ad16-40f4317e4e9a None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:28.522221 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca4444-d9be-4061-ad16-40f4317e4e9a None admin] RBAC: Adding query filter params (name=network) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:28.522717 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca4444-d9be-4061-ad16-40f4317e4e9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:28.524272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca4444-d9be-4061-ad16-40f4317e4e9a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:28.533296 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 351/718] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Jul 14 10:01:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:28.538004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881e174-ad9c-47ee-9b83-653417bc16a7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:28.687220 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:28.693691 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8881e174-ad9c-47ee-9b83-653417bc16a7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:28.736569 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58614c15-b055-4649-9e88-eb071268a76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:28.737189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58614c15-b055-4649-9e88-eb071268a76a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:28.737473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58614c15-b055-4649-9e88-eb071268a76a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:28.737748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58614c15-b055-4649-9e88-eb071268a76a None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:28.738467 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58614c15-b055-4649-9e88-eb071268a76a None admin] RBAC: Adding query filter params (type=network) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:28.738774 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58614c15-b055-4649-9e88-eb071268a76a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:28.740184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58614c15-b055-4649-9e88-eb071268a76a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:28.745400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 368/719] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Jul 14 10:01:28 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:29.387034 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5c374-d5dc-476f-a474-71a62b6f53c2 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:29.387989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5c374-d5dc-476f-a474-71a62b6f53c2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:29.388467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5c374-d5dc-476f-a474-71a62b6f53c2 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:29.390346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 352/720] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:29.395673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c282c12-2f6b-40a1-ae42-d167a0eece59 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:29.396181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c282c12-2f6b-40a1-ae42-d167a0eece59 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:29.396706 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c282c12-2f6b-40a1-ae42-d167a0eece59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:29.418012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5c282c12-2f6b-40a1-ae42-d167a0eece59 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:29.419515 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5c282c12-2f6b-40a1-ae42-d167a0eece59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:29.516060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 369/721] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:29.562326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5c374-d5dc-476f-a474-71a62b6f53c2 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:29.661860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:29.686832 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb7ce0-7933-4440-9a64-b84bc6b21d90 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:29.687179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb7ce0-7933-4440-9a64-b84bc6b21d90 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:29.687471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb7ce0-7933-4440-9a64-b84bc6b21d90 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:29.689132 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 370/722] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:29.695102 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddaa5ef-0716-429a-8993-8a990cd4d25c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:29.695396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddaa5ef-0716-429a-8993-8a990cd4d25c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:29.695659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddaa5ef-0716-429a-8993-8a990cd4d25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:29.701435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5c374-d5dc-476f-a474-71a62b6f53c2 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:29.726065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4ddaa5ef-0716-429a-8993-8a990cd4d25c None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:29.727280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4ddaa5ef-0716-429a-8993-8a990cd4d25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:29.748625 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa61fe-cab9-412d-9b62-a878ae1e851b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:29.750165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa61fe-cab9-412d-9b62-a878ae1e851b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:29.750659 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa61fe-cab9-412d-9b62-a878ae1e851b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:29.751137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa61fe-cab9-412d-9b62-a878ae1e851b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:29.773097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa61fe-cab9-412d-9b62-a878ae1e851b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-52aa61fe-cab9-412d-9b62-a878ae1e851b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:29.782327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:29.783285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:29.784186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:29.784186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:29.784186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:29.784186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:29.784186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:29.784186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:29.784186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:29.784186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:29.784186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:29.784186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:29.784186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:01:29.784186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:29.790632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 353/723] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:01:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:29.796590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa61fe-cab9-412d-9b62-a878ae1e851b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:29.829446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 371/724] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:29.864136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddaa5ef-0716-429a-8993-8a990cd4d25c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:29.890083 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:29.973029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa61fe-cab9-412d-9b62-a878ae1e851b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:29.990199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddaa5ef-0716-429a-8993-8a990cd4d25c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:30.007478 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:30.016445 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b79e3-70af-482a-aa70-3837e58ccad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:30.017155 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b79e3-70af-482a-aa70-3837e58ccad1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:30.017434 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b79e3-70af-482a-aa70-3837e58ccad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:30.017710 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2b79e3-70af-482a-aa70-3837e58ccad1 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:30.018459 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b79e3-70af-482a-aa70-3837e58ccad1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:30.020110 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b79e3-70af-482a-aa70-3837e58ccad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:30.020110 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2b79e3-70af-482a-aa70-3837e58ccad1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:30.038601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 354/725] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:01:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:30.044430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b79e3-70af-482a-aa70-3837e58ccad1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:30.066103 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b9f98-6063-4607-80ea-60c543a880f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:30.067186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b9f98-6063-4607-80ea-60c543a880f5 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:30.067696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b9f98-6063-4607-80ea-60c543a880f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:30.068185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13b9f98-6063-4607-80ea-60c543a880f5 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:30.069186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b9f98-6063-4607-80ea-60c543a880f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:30.070765 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13b9f98-6063-4607-80ea-60c543a880f5 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:30.101280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 372/726] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Jul 14 10:01:29 2022] POST /identity/v3/services => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:01:30.194537 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2b79e3-70af-482a-aa70-3837e58ccad1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:30.244896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d360e196-c7b1-49c5-8661-a69524996783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:30.246064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360e196-c7b1-49c5-8661-a69524996783 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:30.246587 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360e196-c7b1-49c5-8661-a69524996783 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:30.247104 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d360e196-c7b1-49c5-8661-a69524996783 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:30.269121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360e196-c7b1-49c5-8661-a69524996783 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:30.271212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d360e196-c7b1-49c5-8661-a69524996783 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:30.299861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-d360e196-c7b1-49c5-8661-a69524996783 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:30.299861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:30.299861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:30.299861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:30.299861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:30.299861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:30.299861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:30.299861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:30.299861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:30.299861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:30.299861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:30.299861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:30.299861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:30.299861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:30.299861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:30.300788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:30.302739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:30.302739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:30.302739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:30.302739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:30.302739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 355/727] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:30.307811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b9f98-6063-4607-80ea-60c543a880f5 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:30.319984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eea956-d77d-4d10-95ec-9fe2a987a8da None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:30.320329 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eea956-d77d-4d10-95ec-9fe2a987a8da None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:30.320534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eea956-d77d-4d10-95ec-9fe2a987a8da None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:30.325230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 356/728] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:30.330043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce660a-9f23-4722-b5a7-c9edcae57536 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:30.330327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce660a-9f23-4722-b5a7-c9edcae57536 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:30.330606 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce660a-9f23-4722-b5a7-c9edcae57536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:30.361486 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-01ce660a-9f23-4722-b5a7-c9edcae57536 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:30.362718 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-01ce660a-9f23-4722-b5a7-c9edcae57536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:30.431306 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:30.445306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:30.457081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13b9f98-6063-4607-80ea-60c543a880f5 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:30.475880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 357/729] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:30 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 14 10:01:30.497360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05193a3-649c-4b34-95dd-b46b14cb5d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:30.498093 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05193a3-649c-4b34-95dd-b46b14cb5d16 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:30.498377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05193a3-649c-4b34-95dd-b46b14cb5d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:30.498713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05193a3-649c-4b34-95dd-b46b14cb5d16 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:30.499458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05193a3-649c-4b34-95dd-b46b14cb5d16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:30.499767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05193a3-649c-4b34-95dd-b46b14cb5d16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:30.501247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05193a3-649c-4b34-95dd-b46b14cb5d16 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:30.502454 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce660a-9f23-4722-b5a7-c9edcae57536 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:30.519141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 373/730] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:30.526325 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05193a3-649c-4b34-95dd-b46b14cb5d16 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:30.640743 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce660a-9f23-4722-b5a7-c9edcae57536 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:30.656124 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05193a3-649c-4b34-95dd-b46b14cb5d16 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:30.694885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab36c80-5b6f-47f5-8cab-ef6357030059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:30.695679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab36c80-5b6f-47f5-8cab-ef6357030059 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:30.695955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab36c80-5b6f-47f5-8cab-ef6357030059 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:30.696223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab36c80-5b6f-47f5-8cab-ef6357030059 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:30.711957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63d2df-c2de-4b38-8b7c-aff0d6d26938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:30.712755 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63d2df-c2de-4b38-8b7c-aff0d6d26938 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:30.713059 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63d2df-c2de-4b38-8b7c-aff0d6d26938 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:30.713327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63d2df-c2de-4b38-8b7c-aff0d6d26938 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:30.717166 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab36c80-5b6f-47f5-8cab-ef6357030059 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-4ab36c80-5b6f-47f5-8cab-ef6357030059 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:30.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:30.726366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:30.727197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:30.727197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:30.727197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:30.727197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:30.727197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:30.727197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:30.727197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:30.727197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:30.727197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:30.727197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:30.727197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:01:30.727197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:30.733446 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 358/731] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:01:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:30.739546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab36c80-5b6f-47f5-8cab-ef6357030059 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:30.740667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63d2df-c2de-4b38-8b7c-aff0d6d26938 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:30.742196 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63d2df-c2de-4b38-8b7c-aff0d6d26938 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-db63d2df-c2de-4b38-8b7c-aff0d6d26938 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:30.765535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:30.767885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:30.770660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:30.770660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:30.770660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:30.770660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:30.770660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:30.770660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:30.770660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:30.770660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:30.770660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:01:30.770660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:30.770660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 374/732] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:01:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:30.777533 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63d2df-c2de-4b38-8b7c-aff0d6d26938 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:30.781895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:30.808504 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:30.882604 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab36c80-5b6f-47f5-8cab-ef6357030059 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:30.918299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7fcdfb-5f8f-4178-ba29-39c2382b6c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:30.918967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fcdfb-5f8f-4178-ba29-39c2382b6c6c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:30.919244 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fcdfb-5f8f-4178-ba29-39c2382b6c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:30.919514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7fcdfb-5f8f-4178-ba29-39c2382b6c6c None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:30.920256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fcdfb-5f8f-4178-ba29-39c2382b6c6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:30.920558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fcdfb-5f8f-4178-ba29-39c2382b6c6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:30.921946 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7fcdfb-5f8f-4178-ba29-39c2382b6c6c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:30.931361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 359/733] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:01:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:30.932673 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfd30e-7bb5-42cc-94b6-d82e8697d959 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:30.937241 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfd30e-7bb5-42cc-94b6-d82e8697d959 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:30.937566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bfd30e-7bb5-42cc-94b6-d82e8697d959 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:30.939157 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 360/734] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:30.939882 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfd30e-7bb5-42cc-94b6-d82e8697d959 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:30.950150 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63d2df-c2de-4b38-8b7c-aff0d6d26938 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:30.985660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412b705e-0b76-49ed-8efe-d5851b487417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:30.986336 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b705e-0b76-49ed-8efe-d5851b487417 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:30.986619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b705e-0b76-49ed-8efe-d5851b487417 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:30.986892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412b705e-0b76-49ed-8efe-d5851b487417 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:30.987739 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b705e-0b76-49ed-8efe-d5851b487417 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:30.988173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b705e-0b76-49ed-8efe-d5851b487417 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:30.989752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412b705e-0b76-49ed-8efe-d5851b487417 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:31.013247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 375/735] 10.222.0.61 () {56 vars in 1135 bytes} [Thu Jul 14 10:01:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:31.014641 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4ca9f-3de3-4574-b01a-03ffd034d886 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:31.014932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4ca9f-3de3-4574-b01a-03ffd034d886 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:31.015213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4ca9f-3de3-4574-b01a-03ffd034d886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:31.032896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2fe4ca9f-3de3-4574-b01a-03ffd034d886 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:31.034241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2fe4ca9f-3de3-4574-b01a-03ffd034d886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:31.044732 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:31.053560 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bfd30e-7bb5-42cc-94b6-d82e8697d959 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:31.090008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05786f0e-40e1-41f8-a4eb-8992ea936137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:31.090849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05786f0e-40e1-41f8-a4eb-8992ea936137 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:31.091157 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05786f0e-40e1-41f8-a4eb-8992ea936137 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:31.091488 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05786f0e-40e1-41f8-a4eb-8992ea936137 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:31.095359 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:31.102476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05786f0e-40e1-41f8-a4eb-8992ea936137 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:31.103975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05786f0e-40e1-41f8-a4eb-8992ea936137 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:31.113255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 376/736] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:31.114171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4ca9f-3de3-4574-b01a-03ffd034d886 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:31.122387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-05786f0e-40e1-41f8-a4eb-8992ea936137 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:31.122387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:31.122387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:31.122387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:31.122387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:31.122387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:31.122387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:31.122387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:31.122387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:31.122387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:31.122387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:31.122387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:31.122387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:31.122387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:31.122387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:31.124407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:31.126651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:31.126651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:31.126651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:31.126651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:31.126651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 361/737] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:31.132947 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05786f0e-40e1-41f8-a4eb-8992ea936137 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:31.241372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05786f0e-40e1-41f8-a4eb-8992ea936137 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:31.243563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4ca9f-3de3-4574-b01a-03ffd034d886 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:31.274977 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24627b9c-fe0e-4e55-8c2c-796f469512ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:31.275639 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24627b9c-fe0e-4e55-8c2c-796f469512ef None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:31.275915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24627b9c-fe0e-4e55-8c2c-796f469512ef None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:31.276186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24627b9c-fe0e-4e55-8c2c-796f469512ef None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:31.277030 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24627b9c-fe0e-4e55-8c2c-796f469512ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:31.277428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24627b9c-fe0e-4e55-8c2c-796f469512ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:31.278879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24627b9c-fe0e-4e55-8c2c-796f469512ef None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:31.279613 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764488f9-10fd-4c63-90e8-6bb7b3d62dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:31.280266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764488f9-10fd-4c63-90e8-6bb7b3d62dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:31.280546 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764488f9-10fd-4c63-90e8-6bb7b3d62dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:31.280820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764488f9-10fd-4c63-90e8-6bb7b3d62dd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:31.296387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 362/738] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:31.297335 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24627b9c-fe0e-4e55-8c2c-796f469512ef None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:31.306843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:31.356570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:31.367768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764488f9-10fd-4c63-90e8-6bb7b3d62dd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:31.369376 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764488f9-10fd-4c63-90e8-6bb7b3d62dd4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:31.431051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24627b9c-fe0e-4e55-8c2c-796f469512ef None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:31.474895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 377/739] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:31 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 14 10:01:31.475934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764488f9-10fd-4c63-90e8-6bb7b3d62dd4 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:31.482285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbbe6f-c657-46a0-a0f1-ddfdeb259af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:31.483089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbbe6f-c657-46a0-a0f1-ddfdeb259af3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:31.483374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbbe6f-c657-46a0-a0f1-ddfdeb259af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:31.483650 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dbbe6f-c657-46a0-a0f1-ddfdeb259af3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:31.501828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dbbe6f-c657-46a0-a0f1-ddfdeb259af3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-f5dbbe6f-c657-46a0-a0f1-ddfdeb259af3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:31.513614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:31.514746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:31.515794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:31.515794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:31.515794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:31.515794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:31.515794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:31.515794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:31.515794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:31.515794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:31.515794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:31.515794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:31.515794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:31.515794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:31.518955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 363/740] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:31.519939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbbe6f-c657-46a0-a0f1-ddfdeb259af3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:31.589167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:31.597222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:31.598633 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764488f9-10fd-4c63-90e8-6bb7b3d62dd4 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:31.634825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab6838-6b68-40fb-8745-c27ae1ce2177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:31.635629 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab6838-6b68-40fb-8745-c27ae1ce2177 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:31.635909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab6838-6b68-40fb-8745-c27ae1ce2177 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:31.636182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab6838-6b68-40fb-8745-c27ae1ce2177 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:31.636278 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dbbe6f-c657-46a0-a0f1-ddfdeb259af3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:31.644285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab6838-6b68-40fb-8745-c27ae1ce2177 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:31.645836 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab6838-6b68-40fb-8745-c27ae1ce2177 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-ccab6838-6b68-40fb-8745-c27ae1ce2177 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:31.662785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:31.663934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:31.665095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:31.665095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:31.665095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:31.665095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:31.665095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:31.665095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:31.665095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:31.665095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:31.665095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:01:31.665095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:31.666280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 378/741] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Jul 14 10:01:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:31.667080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab6838-6b68-40fb-8745-c27ae1ce2177 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:31.680054 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e9178d-952e-45d6-9f0e-70fe3e5202d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:31.681521 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9178d-952e-45d6-9f0e-70fe3e5202d3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:31.681965 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9178d-952e-45d6-9f0e-70fe3e5202d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:31.682384 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e9178d-952e-45d6-9f0e-70fe3e5202d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:31.696417 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9178d-952e-45d6-9f0e-70fe3e5202d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:31.698800 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e9178d-952e-45d6-9f0e-70fe3e5202d3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:31.719112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-27e9178d-952e-45d6-9f0e-70fe3e5202d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:31.719112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:31.719112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:31.719112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:31.719112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:31.719112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:31.719112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:31.719112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:31.719112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:31.719112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:31.719112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:31.719112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:31.719112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:31.719112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:31.719112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:31.720216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:31.721361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:31.721361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:31.721361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:31.721361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:31.722897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 364/742] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:31.724908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1527da-c970-4c66-bb3a-42ba086ef5ca None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:31.725214 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1527da-c970-4c66-bb3a-42ba086ef5ca None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:31.725489 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1527da-c970-4c66-bb3a-42ba086ef5ca None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:31.726734 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 365/743] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:31.727556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1527da-c970-4c66-bb3a-42ba086ef5ca None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:31.779394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab6838-6b68-40fb-8745-c27ae1ce2177 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:31.817837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd53f99-3467-437a-98f5-fc41fd6f9885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:31.818763 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd53f99-3467-437a-98f5-fc41fd6f9885 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:31.819191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd53f99-3467-437a-98f5-fc41fd6f9885 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:31.819610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd53f99-3467-437a-98f5-fc41fd6f9885 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:31.820589 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd53f99-3467-437a-98f5-fc41fd6f9885 None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:31.821078 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd53f99-3467-437a-98f5-fc41fd6f9885 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:31.822773 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd53f99-3467-437a-98f5-fc41fd6f9885 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:31.830998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 379/744] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:31.831943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd53f99-3467-437a-98f5-fc41fd6f9885 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:31.837539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1527da-c970-4c66-bb3a-42ba086ef5ca None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:31.848476 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:31.859751 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:31.878383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b22822-1a06-43d5-b15f-e4c730f98681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:31.879052 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b22822-1a06-43d5-b15f-e4c730f98681 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:31.879961 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b22822-1a06-43d5-b15f-e4c730f98681 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:31.880230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b22822-1a06-43d5-b15f-e4c730f98681 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:31.881209 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b22822-1a06-43d5-b15f-e4c730f98681 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:31.881513 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b22822-1a06-43d5-b15f-e4c730f98681 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:31.882928 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b22822-1a06-43d5-b15f-e4c730f98681 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:31.906165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 366/745] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Jul 14 10:01:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:31.906165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58123f-ad20-4e5a-9b8f-e7e9d7493690 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:31.906165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58123f-ad20-4e5a-9b8f-e7e9d7493690 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:31.906165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe58123f-ad20-4e5a-9b8f-e7e9d7493690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:31.919616 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-fe58123f-ad20-4e5a-9b8f-e7e9d7493690 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:31.920901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fe58123f-ad20-4e5a-9b8f-e7e9d7493690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:31.940808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd53f99-3467-437a-98f5-fc41fd6f9885 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:31.976370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edde0b-2508-4256-906e-3055237cacc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:31.977081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edde0b-2508-4256-906e-3055237cacc7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:31.977366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edde0b-2508-4256-906e-3055237cacc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:31.977640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19edde0b-2508-4256-906e-3055237cacc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:31.978363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edde0b-2508-4256-906e-3055237cacc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:31.978667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edde0b-2508-4256-906e-3055237cacc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:31.980031 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19edde0b-2508-4256-906e-3055237cacc7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:31.992665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 380/746] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:31.993079 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 367/747] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:31 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 14 10:01:31.993515 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edde0b-2508-4256-906e-3055237cacc7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:31.994302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58123f-ad20-4e5a-9b8f-e7e9d7493690 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:32.082139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:32.108151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe58123f-ad20-4e5a-9b8f-e7e9d7493690 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:32.108335 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19edde0b-2508-4256-906e-3055237cacc7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:32.116388 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:32.147161 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae4e3f-9300-4324-a54e-9824250d385c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:32.148032 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae4e3f-9300-4324-a54e-9824250d385c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:32.148362 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae4e3f-9300-4324-a54e-9824250d385c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:32.148637 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae4e3f-9300-4324-a54e-9824250d385c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:32.160253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae4e3f-9300-4324-a54e-9824250d385c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:32.161842 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dae4e3f-9300-4324-a54e-9824250d385c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:32.181769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-2dae4e3f-9300-4324-a54e-9824250d385c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:32.181769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:32.181769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:32.181769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:32.181769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:32.181769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:32.181769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:32.181769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:32.181769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:32.181769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:32.181769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:32.181769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:32.181769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:32.181769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:32.181769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:32.182758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:32.188856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:32.188856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:32.188856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:32.188856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:32.188856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 368/748] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:32.188856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae4e3f-9300-4324-a54e-9824250d385c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:32.264723 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473b3f8-09ca-473c-98c7-88e0f3fc4db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:32.265558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473b3f8-09ca-473c-98c7-88e0f3fc4db9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:32.265836 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473b3f8-09ca-473c-98c7-88e0f3fc4db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:32.266109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473b3f8-09ca-473c-98c7-88e0f3fc4db9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:32.277759 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473b3f8-09ca-473c-98c7-88e0f3fc4db9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:32.279195 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473b3f8-09ca-473c-98c7-88e0f3fc4db9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:32.296289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dae4e3f-9300-4324-a54e-9824250d385c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:32.297561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-1473b3f8-09ca-473c-98c7-88e0f3fc4db9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:32.297561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:32.297561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:32.297561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:32.297561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:32.297561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:32.297561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:32.297561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:32.297561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:32.297561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:32.297561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:32.297561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:32.297561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:32.297561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:32.297561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:32.298435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:32.299317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:32.299317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:32.299317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:32.299317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:32.299317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 381/749] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:32.299600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473b3f8-09ca-473c-98c7-88e0f3fc4db9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:32.338920 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fab204-1347-4a98-8cee-4ba54e027a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:32.339633 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fab204-1347-4a98-8cee-4ba54e027a0e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:32.339911 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fab204-1347-4a98-8cee-4ba54e027a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:32.340179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fab204-1347-4a98-8cee-4ba54e027a0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:32.347353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fab204-1347-4a98-8cee-4ba54e027a0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:32.348980 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fab204-1347-4a98-8cee-4ba54e027a0e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:32.365456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-72fab204-1347-4a98-8cee-4ba54e027a0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:32.365456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:32.365456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:32.365456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:32.365456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:32.365456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:32.365456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:32.365456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:32.365456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:32.365456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:32.365456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:32.365456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:32.365456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:32.365456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:32.365456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:32.366390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:32.367295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:32.367295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:32.367295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:32.367295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:32.367295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:32.367295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:32.367295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:32.367295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:32.367295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 369/750] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:32.368159 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fab204-1347-4a98-8cee-4ba54e027a0e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:32.388638 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:32.411685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473b3f8-09ca-473c-98c7-88e0f3fc4db9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:32.438471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:32.449600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4251e-f5d0-478f-a7a1-822b604266f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:32.450266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4251e-f5d0-478f-a7a1-822b604266f5 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:32.450668 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4251e-f5d0-478f-a7a1-822b604266f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:32.450937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4251e-f5d0-478f-a7a1-822b604266f5 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:32.451664 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4251e-f5d0-478f-a7a1-822b604266f5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:32.453170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f4251e-f5d0-478f-a7a1-822b604266f5 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:32.473308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 382/751] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Jul 14 10:01:32 2022] POST /identity/v3/services => generated 240 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:01:32.473308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4251e-f5d0-478f-a7a1-822b604266f5 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:32.490766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fab204-1347-4a98-8cee-4ba54e027a0e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:32.533563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf7614-ca76-4fd5-9ffa-24ade208d933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:32.534289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf7614-ca76-4fd5-9ffa-24ade208d933 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:32.534576 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf7614-ca76-4fd5-9ffa-24ade208d933 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:32.534851 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf7614-ca76-4fd5-9ffa-24ade208d933 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:32.536354 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf7614-ca76-4fd5-9ffa-24ade208d933 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:32.536972 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf7614-ca76-4fd5-9ffa-24ade208d933 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:32.539655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf7614-ca76-4fd5-9ffa-24ade208d933 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:32.561293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 370/752] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:32.562304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf7614-ca76-4fd5-9ffa-24ade208d933 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:32.605459 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4251e-f5d0-478f-a7a1-822b604266f5 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:32.644188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9ed18-df23-40e2-884e-712cebe79f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:32.644885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9ed18-df23-40e2-884e-712cebe79f6d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:32.645191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9ed18-df23-40e2-884e-712cebe79f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:32.645489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f9ed18-df23-40e2-884e-712cebe79f6d None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:32.646218 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9ed18-df23-40e2-884e-712cebe79f6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:32.646522 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9ed18-df23-40e2-884e-712cebe79f6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:32.647884 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f9ed18-df23-40e2-884e-712cebe79f6d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:32.656050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:32.661883 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 383/753] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:32.662806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9ed18-df23-40e2-884e-712cebe79f6d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:32.686118 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf7614-ca76-4fd5-9ffa-24ade208d933 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:32.713446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:32.723196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea959cb-ebd0-49f2-acaf-1b7e07f06e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:32.723980 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea959cb-ebd0-49f2-acaf-1b7e07f06e1f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:32.724265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea959cb-ebd0-49f2-acaf-1b7e07f06e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:32.724535 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea959cb-ebd0-49f2-acaf-1b7e07f06e1f None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:32.725377 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea959cb-ebd0-49f2-acaf-1b7e07f06e1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:32.725739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea959cb-ebd0-49f2-acaf-1b7e07f06e1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:32.727165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea959cb-ebd0-49f2-acaf-1b7e07f06e1f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:32.744886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 371/754] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:32.744886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaa000-22f4-4937-82a1-b98cc499a456 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:32.744886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaa000-22f4-4937-82a1-b98cc499a456 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:32.744886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59aaa000-22f4-4937-82a1-b98cc499a456 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:32.744886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 372/755] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:32.744886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aaa000-22f4-4937-82a1-b98cc499a456 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:32.779899 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f9ed18-df23-40e2-884e-712cebe79f6d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:32.815816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ceba5-3053-418c-a58e-8f2c36c13c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:32.816636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ceba5-3053-418c-a58e-8f2c36c13c87 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:32.816915 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ceba5-3053-418c-a58e-8f2c36c13c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:32.817209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ceba5-3053-418c-a58e-8f2c36c13c87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:32.823962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ceba5-3053-418c-a58e-8f2c36c13c87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:32.825440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ceba5-3053-418c-a58e-8f2c36c13c87 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:32.842348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-ff5ceba5-3053-418c-a58e-8f2c36c13c87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:32.842348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:32.842348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:32.842348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:32.842348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:32.842348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:32.842348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:32.842348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:32.842348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:32.842348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:32.842348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:32.842348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:32.842348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:32.842348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:32.842348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:32.843282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:32.844189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:32.844189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:32.844189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:32.844189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:32.844189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:32.844189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:32.844189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:32.844189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:32.844189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 384/756] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:32.844671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ceba5-3053-418c-a58e-8f2c36c13c87 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:32.866055 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59aaa000-22f4-4937-82a1-b98cc499a456 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:32.902048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8bae6d-07ec-4551-8b16-f30ccd2d66b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:32.902859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bae6d-07ec-4551-8b16-f30ccd2d66b4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:32.903138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bae6d-07ec-4551-8b16-f30ccd2d66b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:32.903406 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8bae6d-07ec-4551-8b16-f30ccd2d66b4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:32.905435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:32.912752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8bae6d-07ec-4551-8b16-f30ccd2d66b4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:32.914231 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8bae6d-07ec-4551-8b16-f30ccd2d66b4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-fe8bae6d-07ec-4551-8b16-f30ccd2d66b4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:32.929719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:32.930995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:32.931945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:32.931945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:32.931945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:32.931945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:32.931945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:32.931945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:32.931945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:32.931945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:32.931945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:01:32.931945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:32.937324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 373/757] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Jul 14 10:01:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:32.939160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208d6b9-840b-48f5-8ee9-67895fd2ecf3 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:32.939596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208d6b9-840b-48f5-8ee9-67895fd2ecf3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:32.939872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208d6b9-840b-48f5-8ee9-67895fd2ecf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:32.954196 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ceba5-3053-418c-a58e-8f2c36c13c87 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:32.959815 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8208d6b9-840b-48f5-8ee9-67895fd2ecf3 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:32.961041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8208d6b9-840b-48f5-8ee9-67895fd2ecf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:32.963775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:32.995107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208eb001-a304-4c67-9216-81f873e6424f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:32.995766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208eb001-a304-4c67-9216-81f873e6424f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:32.996046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208eb001-a304-4c67-9216-81f873e6424f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:32.996321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208eb001-a304-4c67-9216-81f873e6424f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:32.997181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208eb001-a304-4c67-9216-81f873e6424f None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=57662256880f43018039715a755c087c, user.id=d5fa502e8b014c719fcece0a130fdd7e) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:32.997492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208eb001-a304-4c67-9216-81f873e6424f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:32.998910 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208eb001-a304-4c67-9216-81f873e6424f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:33.017008 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 385/758] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:32 2022] GET /identity/v3/role_assignments?user.id=d5fa502e8b014c719fcece0a130fdd7e&scope.project.id=57662256880f43018039715a755c087c&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:33.017008 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208eb001-a304-4c67-9216-81f873e6424f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:33.065414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 374/759] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:33.066446 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208d6b9-840b-48f5-8ee9-67895fd2ecf3 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:33.167185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208eb001-a304-4c67-9216-81f873e6424f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:33.189821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208d6b9-840b-48f5-8ee9-67895fd2ecf3 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:33.219646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:33.221311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d14303-acb0-4b35-983f-354da3ad3acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:33.222197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d14303-acb0-4b35-983f-354da3ad3acf None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:33.222514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d14303-acb0-4b35-983f-354da3ad3acf None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:33.222949 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d14303-acb0-4b35-983f-354da3ad3acf None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:33.223710 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d14303-acb0-4b35-983f-354da3ad3acf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:33.224021 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d14303-acb0-4b35-983f-354da3ad3acf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:33.225492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d14303-acb0-4b35-983f-354da3ad3acf None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:33.240987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 386/760] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:33.240987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d14303-acb0-4b35-983f-354da3ad3acf None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:33.242251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f39427-fe61-4609-b5f9-2dd38de60aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:33.247356 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f39427-fe61-4609-b5f9-2dd38de60aa0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:33.248115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f39427-fe61-4609-b5f9-2dd38de60aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:33.250280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f39427-fe61-4609-b5f9-2dd38de60aa0 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:33.251427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f39427-fe61-4609-b5f9-2dd38de60aa0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:33.251964 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f39427-fe61-4609-b5f9-2dd38de60aa0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:33.254326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f39427-fe61-4609-b5f9-2dd38de60aa0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:33.279496 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:33.285819 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 375/761] 10.222.0.61 () {56 vars in 1149 bytes} [Thu Jul 14 10:01:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:33.285819 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f39427-fe61-4609-b5f9-2dd38de60aa0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:33.393060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d14303-acb0-4b35-983f-354da3ad3acf None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:33.404758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f39427-fe61-4609-b5f9-2dd38de60aa0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:33.439977 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf003503-1da3-427b-8af1-6e632b046e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:33.440714 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf003503-1da3-427b-8af1-6e632b046e3a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:33.441022 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf003503-1da3-427b-8af1-6e632b046e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:33.441825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf003503-1da3-427b-8af1-6e632b046e3a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:33.442494 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf003503-1da3-427b-8af1-6e632b046e3a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:33.444037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf003503-1da3-427b-8af1-6e632b046e3a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:33.445454 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc0feb-e77c-4992-8314-ddc0662d6b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:33.447260 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc0feb-e77c-4992-8314-ddc0662d6b19 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:33.447751 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc0feb-e77c-4992-8314-ddc0662d6b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:33.448502 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc0feb-e77c-4992-8314-ddc0662d6b19 None admin] PATH_INFO: `/v3/projects/57662256880f43018039715a755c087c/users/e9f4efa3e2f343c2ac858a277601a7df/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:33.450671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc0feb-e77c-4992-8314-ddc0662d6b19 None admin] RBAC: Authorizing `identity:create_grant(project_id=57662256880f43018039715a755c087c, user_id=e9f4efa3e2f343c2ac858a277601a7df, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:33.452256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc0feb-e77c-4992-8314-ddc0662d6b19 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-cf003503-1da3-427b-8af1-6e632b046e3a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:33.458295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:33.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:33.460525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:33.460525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:33.460525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:33.460525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:33.460525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:33.460525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:33.460525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:33.460525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:33.460525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:33.460525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:33.460525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:33.460525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:33.460525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:33.460525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:01:33.460525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:33.465077 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 387/762] 10.222.0.61 () {56 vars in 1129 bytes} [Thu Jul 14 10:01:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:33.465077 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf003503-1da3-427b-8af1-6e632b046e3a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:33.477183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 376/763] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:01:33 2022] PUT /identity/v3/projects/57662256880f43018039715a755c087c/users/e9f4efa3e2f343c2ac858a277601a7df/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:01:33.478596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc0feb-e77c-4992-8314-ddc0662d6b19 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:33.578464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:33.629867 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:33.637034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf003503-1da3-427b-8af1-6e632b046e3a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:33.678155 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4493445d-f90e-4988-9a1b-91caa7786780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:33.678378 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493445d-f90e-4988-9a1b-91caa7786780 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:33.678459 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493445d-f90e-4988-9a1b-91caa7786780 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:33.678741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4493445d-f90e-4988-9a1b-91caa7786780 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:33.689812 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc0feb-e77c-4992-8314-ddc0662d6b19 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:33.731954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd67765-d180-429b-9d69-c77c5d0feccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:33.733011 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd67765-d180-429b-9d69-c77c5d0feccd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:33.733531 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd67765-d180-429b-9d69-c77c5d0feccd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:33.734006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd67765-d180-429b-9d69-c77c5d0feccd None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:33.734989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd67765-d180-429b-9d69-c77c5d0feccd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:33.735491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd67765-d180-429b-9d69-c77c5d0feccd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:33.737919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd67765-d180-429b-9d69-c77c5d0feccd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:33.744270 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 377/764] 10.222.0.61 () {56 vars in 1124 bytes} [Thu Jul 14 10:01:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:33.750430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd67765-d180-429b-9d69-c77c5d0feccd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:33.752471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4493445d-f90e-4988-9a1b-91caa7786780 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:33.754559 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4493445d-f90e-4988-9a1b-91caa7786780 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:33.849024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:33.856810 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 388/765] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:01:33.869960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4493445d-f90e-4988-9a1b-91caa7786780 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:33.890589 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd67765-d180-429b-9d69-c77c5d0feccd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:33.900823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:33.927819 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c60c2-6251-4fc6-996b-82263fcbe55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:33.928815 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c60c2-6251-4fc6-996b-82263fcbe55e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:33.929320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c60c2-6251-4fc6-996b-82263fcbe55e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:33.929902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3c60c2-6251-4fc6-996b-82263fcbe55e None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:33.931082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c60c2-6251-4fc6-996b-82263fcbe55e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:33.931584 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c60c2-6251-4fc6-996b-82263fcbe55e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:33.933281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3c60c2-6251-4fc6-996b-82263fcbe55e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:33.944894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 378/766] 10.222.0.61 () {56 vars in 1124 bytes} [Thu Jul 14 10:01:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:33.946394 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c60c2-6251-4fc6-996b-82263fcbe55e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:34.069866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4493445d-f90e-4988-9a1b-91caa7786780 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:34.116952 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3c60c2-6251-4fc6-996b-82263fcbe55e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:34.132981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9955a3e-6882-4321-98e4-cacde71f7d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:34.134206 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9955a3e-6882-4321-98e4-cacde71f7d1f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:34.135789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9955a3e-6882-4321-98e4-cacde71f7d1f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:34.136510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9955a3e-6882-4321-98e4-cacde71f7d1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:34.155589 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9955a3e-6882-4321-98e4-cacde71f7d1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:34.158348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9955a3e-6882-4321-98e4-cacde71f7d1f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:34.188726 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0b6db-1b98-48c8-9701-0f8e22c34d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:34.189765 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0b6db-1b98-48c8-9701-0f8e22c34d96 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:34.190331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0b6db-1b98-48c8-9701-0f8e22c34d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:34.190799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0b6db-1b98-48c8-9701-0f8e22c34d96 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:34.191826 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0b6db-1b98-48c8-9701-0f8e22c34d96 None admin] RBAC: Adding query filter params (interface=public, service_id=3595d3b7daac4a638fd3d30b5a186239, region_id=RegionOne) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:34.192349 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0b6db-1b98-48c8-9701-0f8e22c34d96 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:34.193953 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f0b6db-1b98-48c8-9701-0f8e22c34d96 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:34.196188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-c9955a3e-6882-4321-98e4-cacde71f7d1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:34.196188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:34.196188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:34.196188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:34.196188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:34.196188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:34.196188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:34.196188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:34.196188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:34.196188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:34.196188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:34.196188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:34.196188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:34.196188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:34.196188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:34.197101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:34.198017 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:34.198017 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:34.198017 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:34.198017 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:34.203181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 389/767] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:34.211757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9955a3e-6882-4321-98e4-cacde71f7d1f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:34.214634 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 379/768] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Jul 14 10:01:33 2022] GET /identity/v3/endpoints?service_id=3595d3b7daac4a638fd3d30b5a186239&interface=public®ion_id=RegionOne => generated 194 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:34.216832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0b6db-1b98-48c8-9701-0f8e22c34d96 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:34.282948 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:34.311154 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:34.398056 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0b6db-1b98-48c8-9701-0f8e22c34d96 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:34.441975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d9bac-cde3-4795-bec1-5d95108c0c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:34.442648 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d9bac-cde3-4795-bec1-5d95108c0c36 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:34.442928 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d9bac-cde3-4795-bec1-5d95108c0c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:34.443202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d9bac-cde3-4795-bec1-5d95108c0c36 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:34.443890 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d9bac-cde3-4795-bec1-5d95108c0c36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:34.445293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d9bac-cde3-4795-bec1-5d95108c0c36 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:34.447055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9955a3e-6882-4321-98e4-cacde71f7d1f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:34.460318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 380/769] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Jul 14 10:01:34 2022] GET /identity/v3/services => generated 772 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:34.496615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fb365-6426-4183-a8ac-53f551e0ecda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:34.497372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fb365-6426-4183-a8ac-53f551e0ecda None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:34.497652 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fb365-6426-4183-a8ac-53f551e0ecda None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:34.497922 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fb365-6426-4183-a8ac-53f551e0ecda None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:34.498777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509fb365-6426-4183-a8ac-53f551e0ecda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:34.499086 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509fb365-6426-4183-a8ac-53f551e0ecda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:34.500453 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509fb365-6426-4183-a8ac-53f551e0ecda None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:34.513292 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 390/770] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:34.518487 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d9bac-cde3-4795-bec1-5d95108c0c36 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:34.625360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0c76c-9642-4818-ad0a-b6b8f5935ebf None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:34.625683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0c76c-9642-4818-ad0a-b6b8f5935ebf None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:34.625909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0c76c-9642-4818-ad0a-b6b8f5935ebf None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:34.627532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 391/771] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:34.638408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fdd76-895e-4f5e-8e20-de86f9a6669a None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:34.639055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fdd76-895e-4f5e-8e20-de86f9a6669a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:34.639552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1fdd76-895e-4f5e-8e20-de86f9a6669a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:34.643120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:34.649807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d9bac-cde3-4795-bec1-5d95108c0c36 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:34.674990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-bf1fdd76-895e-4f5e-8e20-de86f9a6669a None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:34.676235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bf1fdd76-895e-4f5e-8e20-de86f9a6669a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:34.698136 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6f55f-d1f8-471a-9005-7262e8f7e5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:34.699290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6f55f-d1f8-471a-9005-7262e8f7e5cb None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:34.699774 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6f55f-d1f8-471a-9005-7262e8f7e5cb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:34.700277 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e6f55f-d1f8-471a-9005-7262e8f7e5cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:34.708829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6f55f-d1f8-471a-9005-7262e8f7e5cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:34.710556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e6f55f-d1f8-471a-9005-7262e8f7e5cb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:34.726984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-f2e6f55f-d1f8-471a-9005-7262e8f7e5cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:34.726984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:34.726984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:34.726984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:34.726984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:34.726984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:34.726984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:34.726984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:34.726984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:34.726984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:34.726984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:34.726984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:34.726984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:34.726984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:34.726984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:34.727909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:34.728793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:34.728793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:34.728793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:34.728793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:34.728793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:34.728793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:34.728793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:34.728793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:34.731428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 381/772] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:34.737092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6f55f-d1f8-471a-9005-7262e8f7e5cb None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:34.752391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 392/773] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:34.788525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fdd76-895e-4f5e-8e20-de86f9a6669a None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:34.849412 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:34.888723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e6f55f-d1f8-471a-9005-7262e8f7e5cb None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:34.958153 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c621ad5-260f-466e-97d6-7a93d8b3bc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:34.958752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c621ad5-260f-466e-97d6-7a93d8b3bc58 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:34.959034 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c621ad5-260f-466e-97d6-7a93d8b3bc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:34.959315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c621ad5-260f-466e-97d6-7a93d8b3bc58 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:34.960122 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c621ad5-260f-466e-97d6-7a93d8b3bc58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:34.960433 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c621ad5-260f-466e-97d6-7a93d8b3bc58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:34.961897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c621ad5-260f-466e-97d6-7a93d8b3bc58 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:34.984461 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1fdd76-895e-4f5e-8e20-de86f9a6669a None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:34.997241 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 382/774] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:35.003376 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c621ad5-260f-466e-97d6-7a93d8b3bc58 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:35.026696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:35.070383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a1b5d-0898-4481-a73b-28d0ac6c7a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:35.071183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a1b5d-0898-4481-a73b-28d0ac6c7a62 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:35.071462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a1b5d-0898-4481-a73b-28d0ac6c7a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:35.071733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a1b5d-0898-4481-a73b-28d0ac6c7a62 None admin] PATH_INFO: `/v3/services/network` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:35.072391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a1b5d-0898-4481-a73b-28d0ac6c7a62 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:35.074060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a1b5d-0898-4481-a73b-28d0ac6c7a62 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-d86a1b5d-0898-4481-a73b-28d0ac6c7a62 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:35.099291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:35.100342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:35.101227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:35.101227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:35.101227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:35.101227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:35.101227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:35.101227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:35.101227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:35.101227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:35.101227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:35.101227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:35.101227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:35.101227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:35.101227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:01:35.101227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:35.102656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 393/775] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:34 2022] GET /identity/v3/services/network => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:35.106299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a1b5d-0898-4481-a73b-28d0ac6c7a62 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:35.174824 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c621ad5-260f-466e-97d6-7a93d8b3bc58 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:35.249014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f7f4b-4928-409f-bffd-f07982f26513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:35.249652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f7f4b-4928-409f-bffd-f07982f26513 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:35.249937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f7f4b-4928-409f-bffd-f07982f26513 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:35.250217 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f7f4b-4928-409f-bffd-f07982f26513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:35.251083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f7f4b-4928-409f-bffd-f07982f26513 None admin] RBAC: Adding query filter params (role.id=ba4088a289ea4b9a9aa9d9cd7a27e731, scope.project.id=57662256880f43018039715a755c087c, user.id=dd2b9ad7efa848938627385b411727ee) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:35.256947 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f7f4b-4928-409f-bffd-f07982f26513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:35.256947 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f7f4b-4928-409f-bffd-f07982f26513 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:35.260173 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 383/776] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:35 2022] GET /identity/v3/role_assignments?user.id=dd2b9ad7efa848938627385b411727ee&scope.project.id=57662256880f43018039715a755c087c&role.id=ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:35.312966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae69b5a-ebf8-4e9d-957c-7082afe5b8a6 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:35.315282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae69b5a-ebf8-4e9d-957c-7082afe5b8a6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:35.315282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae69b5a-ebf8-4e9d-957c-7082afe5b8a6 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:35.317039 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 384/777] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:01:35.319312 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:35.322158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd40724-4af6-4df5-a209-142cff56ce96 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:35.322754 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd40724-4af6-4df5-a209-142cff56ce96 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:35.322862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a1b5d-0898-4481-a73b-28d0ac6c7a62 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:35.323328 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd40724-4af6-4df5-a209-142cff56ce96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:35.343146 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-bcd40724-4af6-4df5-a209-142cff56ce96 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:35.344874 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-bcd40724-4af6-4df5-a209-142cff56ce96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:35.392589 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7de3f1-3261-49cd-adb3-cdeed1ee6dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:35.393321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7de3f1-3261-49cd-adb3-cdeed1ee6dde None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:35.393600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7de3f1-3261-49cd-adb3-cdeed1ee6dde None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:35.393871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7de3f1-3261-49cd-adb3-cdeed1ee6dde None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:35.394652 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7de3f1-3261-49cd-adb3-cdeed1ee6dde None admin] RBAC: Adding query filter params (name=network) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:35.394992 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7de3f1-3261-49cd-adb3-cdeed1ee6dde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:35.396414 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7de3f1-3261-49cd-adb3-cdeed1ee6dde None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:35.407283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 394/778] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Jul 14 10:01:35 2022] GET /identity/v3/services?name=network => generated 124 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:35.414841 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7de3f1-3261-49cd-adb3-cdeed1ee6dde None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:35.485561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 385/779] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:35.526496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd40724-4af6-4df5-a209-142cff56ce96 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:35.546451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:35.591781 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7de3f1-3261-49cd-adb3-cdeed1ee6dde None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:35.659160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49247a36-c99e-4dc2-829b-12ae0739ad52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:35.660757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49247a36-c99e-4dc2-829b-12ae0739ad52 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:35.661207 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49247a36-c99e-4dc2-829b-12ae0739ad52 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:35.661620 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49247a36-c99e-4dc2-829b-12ae0739ad52 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:35.662478 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49247a36-c99e-4dc2-829b-12ae0739ad52 None admin] RBAC: Adding query filter params (type=network) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:35.663710 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49247a36-c99e-4dc2-829b-12ae0739ad52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:35.666186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49247a36-c99e-4dc2-829b-12ae0739ad52 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:35.677143 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 395/780] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Jul 14 10:01:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:35.682951 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd40724-4af6-4df5-a209-142cff56ce96 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:35.687314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49247a36-c99e-4dc2-829b-12ae0739ad52 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:35.723771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:35.793797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05522c88-b698-4b17-89e6-7d0a89bbf576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:35.794463 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05522c88-b698-4b17-89e6-7d0a89bbf576 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:35.794746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05522c88-b698-4b17-89e6-7d0a89bbf576 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:35.795220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05522c88-b698-4b17-89e6-7d0a89bbf576 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:35.798250 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49247a36-c99e-4dc2-829b-12ae0739ad52 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:35.834467 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15105640-3f3c-48d1-a499-574fbe3f44c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:35.835453 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15105640-3f3c-48d1-a499-574fbe3f44c7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:35.835879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15105640-3f3c-48d1-a499-574fbe3f44c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:35.836235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05522c88-b698-4b17-89e6-7d0a89bbf576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:35.836448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15105640-3f3c-48d1-a499-574fbe3f44c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:35.837422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15105640-3f3c-48d1-a499-574fbe3f44c7 None admin] RBAC: Adding query filter params (interface=public, service_id=62f2209e490243adb157b73414a87938, region_id=RegionOne) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:35.837937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15105640-3f3c-48d1-a499-574fbe3f44c7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:35.839446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15105640-3f3c-48d1-a499-574fbe3f44c7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-05522c88-b698-4b17-89e6-7d0a89bbf576 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:35.844095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:35.845273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:35.846289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:35.846289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:35.846289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:35.846289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:35.846289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:35.846289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:35.846289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:35.846289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:35.846289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:35.846289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:35.846289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:01:35.846289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:35.846289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 386/781] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:01:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:35.852386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05522c88-b698-4b17-89e6-7d0a89bbf576 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:35.852570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 396/782] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Jul 14 10:01:35 2022] GET /identity/v3/endpoints?service_id=62f2209e490243adb157b73414a87938&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:35.858154 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15105640-3f3c-48d1-a499-574fbe3f44c7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:35.963432 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:35.973073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15105640-3f3c-48d1-a499-574fbe3f44c7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:35.973321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:36.004842 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05522c88-b698-4b17-89e6-7d0a89bbf576 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:36.019691 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414688df-9de9-46ce-a1c8-a6c5d8f92976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:36.020766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414688df-9de9-46ce-a1c8-a6c5d8f92976 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:36.021512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414688df-9de9-46ce-a1c8-a6c5d8f92976 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:36.021966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414688df-9de9-46ce-a1c8-a6c5d8f92976 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:36.023170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414688df-9de9-46ce-a1c8-a6c5d8f92976 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:36.024673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414688df-9de9-46ce-a1c8-a6c5d8f92976 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:36.032856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 397/783] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Jul 14 10:01:35 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:36.034752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd6ccd-be3e-4f30-b5c9-284c13c78cf9 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:36.035608 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd6ccd-be3e-4f30-b5c9-284c13c78cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:36.036325 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd6ccd-be3e-4f30-b5c9-284c13c78cf9 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:36.042100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbc3b4-55fc-45f6-9f89-f16f61b199a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:36.042100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbc3b4-55fc-45f6-9f89-f16f61b199a4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:36.042100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbc3b4-55fc-45f6-9f89-f16f61b199a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:36.042100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbc3b4-55fc-45f6-9f89-f16f61b199a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:36.042554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbc3b4-55fc-45f6-9f89-f16f61b199a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:36.042725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 398/784] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:36.042898 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbc3b4-55fc-45f6-9f89-f16f61b199a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:36.044366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbc3b4-55fc-45f6-9f89-f16f61b199a4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:36.044585 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f2311-0d98-4b2d-a97f-2c464b86d465 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:36.045042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f2311-0d98-4b2d-a97f-2c464b86d465 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:36.045452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77f2311-0d98-4b2d-a97f-2c464b86d465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:36.059948 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 387/785] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:01:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:36.060969 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbc3b4-55fc-45f6-9f89-f16f61b199a4 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:36.068615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f77f2311-0d98-4b2d-a97f-2c464b86d465 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:36.070129 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f77f2311-0d98-4b2d-a97f-2c464b86d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:36.156050 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 399/786] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:36.182822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f2311-0d98-4b2d-a97f-2c464b86d465 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:36.262552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbc3b4-55fc-45f6-9f89-f16f61b199a4 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:36.296818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:36.313245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e87a8-5244-4e52-bd4b-3785bc5e12d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:36.314199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e87a8-5244-4e52-bd4b-3785bc5e12d0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:36.314488 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e87a8-5244-4e52-bd4b-3785bc5e12d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:36.314762 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e87a8-5244-4e52-bd4b-3785bc5e12d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:36.318215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:36.320820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77f2311-0d98-4b2d-a97f-2c464b86d465 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:36.330134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e87a8-5244-4e52-bd4b-3785bc5e12d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:36.333652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e87a8-5244-4e52-bd4b-3785bc5e12d0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:36.360001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-9b0e87a8-5244-4e52-bd4b-3785bc5e12d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:36.360001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:36.360001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:36.360001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:36.360001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:36.360001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:36.360001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:36.360001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:36.360001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:36.360001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:36.360001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:36.360001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:36.360001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:36.360001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:36.360001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:36.361175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:36.362168 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:36.362168 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:36.362168 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:36.362168 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:36.362168 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 388/787] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:36.367458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e87a8-5244-4e52-bd4b-3785bc5e12d0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:36.369299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8d520-8674-4058-a851-29369ed509d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:36.370566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8d520-8674-4058-a851-29369ed509d9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:36.371955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8d520-8674-4058-a851-29369ed509d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:36.372382 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8d520-8674-4058-a851-29369ed509d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:36.386925 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8d520-8674-4058-a851-29369ed509d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-47d8d520-8674-4058-a851-29369ed509d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:36.397895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:36.399048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:36.399891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:36.399891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:36.399891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:36.399891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:36.399891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:36.399891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:36.399891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:36.399891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:36.399891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:36.399891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:36.399891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:01:36.399891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:36.402983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 400/788] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:01:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:36.409106 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8d520-8674-4058-a851-29369ed509d9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:36.476695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e87a8-5244-4e52-bd4b-3785bc5e12d0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:36.512573 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0314d32-291b-401c-93ff-9f9d658672cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:36.513270 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0314d32-291b-401c-93ff-9f9d658672cc None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:36.513596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0314d32-291b-401c-93ff-9f9d658672cc None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:36.513807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0314d32-291b-401c-93ff-9f9d658672cc None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:36.514530 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0314d32-291b-401c-93ff-9f9d658672cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:36.514835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0314d32-291b-401c-93ff-9f9d658672cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:36.516186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0314d32-291b-401c-93ff-9f9d658672cc None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:36.516546 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8d520-8674-4058-a851-29369ed509d9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:36.533441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 389/789] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:36.535400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0314d32-291b-401c-93ff-9f9d658672cc None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:36.552977 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95af56-f836-45f2-ba93-bff3f345fde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:36.553957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95af56-f836-45f2-ba93-bff3f345fde1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:36.554457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95af56-f836-45f2-ba93-bff3f345fde1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:36.555175 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95af56-f836-45f2-ba93-bff3f345fde1 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:36.556121 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95af56-f836-45f2-ba93-bff3f345fde1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:36.556664 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95af56-f836-45f2-ba93-bff3f345fde1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:36.558248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b95af56-f836-45f2-ba93-bff3f345fde1 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:36.567680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 401/790] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:01:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:36.573318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95af56-f836-45f2-ba93-bff3f345fde1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:36.585625 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:36.623041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:36.698024 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0314d32-291b-401c-93ff-9f9d658672cc None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:36.734598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b95af56-f836-45f2-ba93-bff3f345fde1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:36.759766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b78a8-b62a-44ff-910e-af5b644b8b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:36.760905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b78a8-b62a-44ff-910e-af5b644b8b06 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:36.761424 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b78a8-b62a-44ff-910e-af5b644b8b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:36.761914 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b78a8-b62a-44ff-910e-af5b644b8b06 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:36.770129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b78a8-b62a-44ff-910e-af5b644b8b06 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:36.771843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b78a8-b62a-44ff-910e-af5b644b8b06 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-5c1b78a8-b62a-44ff-910e-af5b644b8b06 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:36.788288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:36.789295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:36.790249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:36.790249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:36.790249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:36.790249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:36.790249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:36.790249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:36.790249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:36.790249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:36.790249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:01:36.790249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:36.792804 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 390/791] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:01:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:36.798340 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b78a8-b62a-44ff-910e-af5b644b8b06 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:36.798870 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784103c-c8bc-4c3a-80fc-13c67d428076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:36.799670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784103c-c8bc-4c3a-80fc-13c67d428076 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:36.800098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784103c-c8bc-4c3a-80fc-13c67d428076 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:36.800379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784103c-c8bc-4c3a-80fc-13c67d428076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:36.823060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784103c-c8bc-4c3a-80fc-13c67d428076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:36.824497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4784103c-c8bc-4c3a-80fc-13c67d428076 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:36.850370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-4784103c-c8bc-4c3a-80fc-13c67d428076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:36.850370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:36.850370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:36.850370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:36.850370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:36.850370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:36.850370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:36.850370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:36.850370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:36.850370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:36.850370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:36.850370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:36.850370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:36.850370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:36.850370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:36.851338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:36.852236 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:36.852236 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:36.852236 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:36.852236 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:36.861085 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 402/792] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:36.869972 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784103c-c8bc-4c3a-80fc-13c67d428076 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:36.908300 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b78a8-b62a-44ff-910e-af5b644b8b06 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:36.916894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:36.945367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:36.993753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784103c-c8bc-4c3a-80fc-13c67d428076 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:37.031423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d868b1f-09ea-4dc8-8b66-2f5ff67d52bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:37.032092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d868b1f-09ea-4dc8-8b66-2f5ff67d52bb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:37.032373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d868b1f-09ea-4dc8-8b66-2f5ff67d52bb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:37.032648 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d868b1f-09ea-4dc8-8b66-2f5ff67d52bb None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:37.033435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d868b1f-09ea-4dc8-8b66-2f5ff67d52bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:37.033744 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d868b1f-09ea-4dc8-8b66-2f5ff67d52bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:37.035175 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d868b1f-09ea-4dc8-8b66-2f5ff67d52bb None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:37.048162 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 403/793] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:37.053675 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807f3c8-9a97-49a5-b438-bfb7ef76ad3f None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:37.053962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807f3c8-9a97-49a5-b438-bfb7ef76ad3f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:37.054243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4807f3c8-9a97-49a5-b438-bfb7ef76ad3f None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:37.056097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 404/794] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:37.057245 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807f3c8-9a97-49a5-b438-bfb7ef76ad3f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:37.067152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07334b05-0953-49dc-b63d-1c3eea03d1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:37.068076 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07334b05-0953-49dc-b63d-1c3eea03d1bc None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:37.068549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07334b05-0953-49dc-b63d-1c3eea03d1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:37.069070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07334b05-0953-49dc-b63d-1c3eea03d1bc None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:37.070075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07334b05-0953-49dc-b63d-1c3eea03d1bc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:37.070602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07334b05-0953-49dc-b63d-1c3eea03d1bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:37.072179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07334b05-0953-49dc-b63d-1c3eea03d1bc None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:37.091522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 391/795] 10.222.0.61 () {56 vars in 1135 bytes} [Thu Jul 14 10:01:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:37.093033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2c6cf-94fd-47ca-9563-be809fd1091c None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:37.093529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2c6cf-94fd-47ca-9563-be809fd1091c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:37.093994 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd2c6cf-94fd-47ca-9563-be809fd1091c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:37.111001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0bd2c6cf-94fd-47ca-9563-be809fd1091c None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:37.112671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0bd2c6cf-94fd-47ca-9563-be809fd1091c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:37.167661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4807f3c8-9a97-49a5-b438-bfb7ef76ad3f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:37.193637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:37.193986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 392/796] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:37.195473 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2c6cf-94fd-47ca-9563-be809fd1091c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:37.204985 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8016bd7-524f-4571-89a2-aa783fcb8cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:37.204985 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8016bd7-524f-4571-89a2-aa783fcb8cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:37.205249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8016bd7-524f-4571-89a2-aa783fcb8cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:37.205470 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8016bd7-524f-4571-89a2-aa783fcb8cf9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:37.214264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8016bd7-524f-4571-89a2-aa783fcb8cf9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:37.215037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8016bd7-524f-4571-89a2-aa783fcb8cf9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-e8016bd7-524f-4571-89a2-aa783fcb8cf9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:37.230120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:37.231293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:37.232422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:37.232422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:37.232422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:37.232422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:37.232422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:37.232422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:37.232422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:37.232422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:37.232422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:01:37.232422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:37.232422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 405/797] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Jul 14 10:01:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:37.232422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8016bd7-524f-4571-89a2-aa783fcb8cf9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:37.327536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:37.337123 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd2c6cf-94fd-47ca-9563-be809fd1091c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:37.361779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8016bd7-524f-4571-89a2-aa783fcb8cf9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:37.376887 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f8cc3-5356-4abc-a902-8cd0c4f9c4e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:37.378374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f8cc3-5356-4abc-a902-8cd0c4f9c4e0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:37.379139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f8cc3-5356-4abc-a902-8cd0c4f9c4e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:37.380120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f8cc3-5356-4abc-a902-8cd0c4f9c4e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:37.395025 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f8cc3-5356-4abc-a902-8cd0c4f9c4e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:37.396714 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f8cc3-5356-4abc-a902-8cd0c4f9c4e0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:37.397556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3098a-fc08-4d93-9f50-b8cff3b6bca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:37.398413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3098a-fc08-4d93-9f50-b8cff3b6bca0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:37.398698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3098a-fc08-4d93-9f50-b8cff3b6bca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:37.398971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3098a-fc08-4d93-9f50-b8cff3b6bca0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:37.399632 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3098a-fc08-4d93-9f50-b8cff3b6bca0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:37.400999 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3098a-fc08-4d93-9f50-b8cff3b6bca0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-93e3098a-fc08-4d93-9f50-b8cff3b6bca0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:37.413956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:37.414903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:37.415747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 406/798] 10.222.0.61 () {56 vars in 1129 bytes} [Thu Jul 14 10:01:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:37.420819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3098a-fc08-4d93-9f50-b8cff3b6bca0 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:37.421225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e84f8cc3-5356-4abc-a902-8cd0c4f9c4e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:37.421225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:37.421225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:37.421225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:37.421225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:37.421225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:37.421225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:37.421225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:37.421225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:37.421225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:37.421225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:37.421225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:37.421225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:37.421225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:37.421225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:37.422068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:37.422957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:37.422957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:37.422957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:37.422957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:37.430807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 393/799] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:37.432497 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f8cc3-5356-4abc-a902-8cd0c4f9c4e0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:37.473952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:37.531415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3098a-fc08-4d93-9f50-b8cff3b6bca0 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:37.568580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74d487-b4e9-465a-a52c-914684203f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:37.569706 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74d487-b4e9-465a-a52c-914684203f33 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:37.570108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74d487-b4e9-465a-a52c-914684203f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:37.570481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74d487-b4e9-465a-a52c-914684203f33 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:37.571598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74d487-b4e9-465a-a52c-914684203f33 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:37.572291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74d487-b4e9-465a-a52c-914684203f33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:37.574128 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74d487-b4e9-465a-a52c-914684203f33 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:37.580016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f8cc3-5356-4abc-a902-8cd0c4f9c4e0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:37.601337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 407/800] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Jul 14 10:01:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:37.601337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74d487-b4e9-465a-a52c-914684203f33 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:37.636209 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fda7b-6bfe-4770-89bd-1b391d880cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:37.637529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fda7b-6bfe-4770-89bd-1b391d880cbe None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:37.638494 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fda7b-6bfe-4770-89bd-1b391d880cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:37.639209 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fda7b-6bfe-4770-89bd-1b391d880cbe None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:37.640169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fda7b-6bfe-4770-89bd-1b391d880cbe None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:37.640674 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fda7b-6bfe-4770-89bd-1b391d880cbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:37.644020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fda7b-6bfe-4770-89bd-1b391d880cbe None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:37.654249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 394/801] 10.222.0.61 () {56 vars in 1124 bytes} [Thu Jul 14 10:01:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:37.655686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fda7b-6bfe-4770-89bd-1b391d880cbe None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:37.670820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:37.717975 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74d487-b4e9-465a-a52c-914684203f33 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:37.719614 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:37.755027 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6dced-8ac5-4891-83ef-9658e180bf87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:37.755685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6dced-8ac5-4891-83ef-9658e180bf87 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:37.755962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6dced-8ac5-4891-83ef-9658e180bf87 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:37.756231 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6dced-8ac5-4891-83ef-9658e180bf87 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:37.756964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6dced-8ac5-4891-83ef-9658e180bf87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:37.757269 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6dced-8ac5-4891-83ef-9658e180bf87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:37.758611 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6dced-8ac5-4891-83ef-9658e180bf87 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:37.765543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fda7b-6bfe-4770-89bd-1b391d880cbe None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:37.771799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 408/802] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:37.773203 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e054948-2db6-48b5-8dc2-2ab4f296a294 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:37.773729 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e054948-2db6-48b5-8dc2-2ab4f296a294 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:37.773998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e054948-2db6-48b5-8dc2-2ab4f296a294 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:37.775371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 409/803] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:37.779726 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e054948-2db6-48b5-8dc2-2ab4f296a294 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:37.806926 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc7727-4c88-4321-a41f-fd74af0deff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:37.807783 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc7727-4c88-4321-a41f-fd74af0deff1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:37.808364 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc7727-4c88-4321-a41f-fd74af0deff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:37.808654 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc7727-4c88-4321-a41f-fd74af0deff1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:37.881669 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc7727-4c88-4321-a41f-fd74af0deff1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:37.883783 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc7727-4c88-4321-a41f-fd74af0deff1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:37.885926 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e054948-2db6-48b5-8dc2-2ab4f296a294 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:37.898063 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:37.929185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b346fc7-567a-44c9-adf9-6beaa1aa6591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:37.930879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b346fc7-567a-44c9-adf9-6beaa1aa6591 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:37.931160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b346fc7-567a-44c9-adf9-6beaa1aa6591 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:37.931432 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b346fc7-567a-44c9-adf9-6beaa1aa6591 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:37.932141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b346fc7-567a-44c9-adf9-6beaa1aa6591 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:37.932442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b346fc7-567a-44c9-adf9-6beaa1aa6591 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:37.933818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b346fc7-567a-44c9-adf9-6beaa1aa6591 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:37.939260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 410/804] 10.222.0.61 () {56 vars in 1124 bytes} [Thu Jul 14 10:01:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:37.940135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b346fc7-567a-44c9-adf9-6beaa1aa6591 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:37.969356 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:38.008543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 395/805] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:01:38.010297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcca794-51b5-417f-8309-a9d90d9aebaf None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:38.010583 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcca794-51b5-417f-8309-a9d90d9aebaf None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:38.010857 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcca794-51b5-417f-8309-a9d90d9aebaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:38.028092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-adcca794-51b5-417f-8309-a9d90d9aebaf None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:38.029400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-adcca794-51b5-417f-8309-a9d90d9aebaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:38.052365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b346fc7-567a-44c9-adf9-6beaa1aa6591 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:38.090266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be280f52-fe8a-4a00-ba01-abba60d59748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:38.091166 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be280f52-fe8a-4a00-ba01-abba60d59748 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:38.091566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be280f52-fe8a-4a00-ba01-abba60d59748 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:38.091841 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be280f52-fe8a-4a00-ba01-abba60d59748 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:38.100971 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 396/806] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:38.100971 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcca794-51b5-417f-8309-a9d90d9aebaf None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:38.110009 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be280f52-fe8a-4a00-ba01-abba60d59748 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:38.111461 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be280f52-fe8a-4a00-ba01-abba60d59748 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:38.127209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-be280f52-fe8a-4a00-ba01-abba60d59748 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:38.127209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:38.127209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:38.127209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:38.127209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:38.127209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:38.127209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:38.127209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:38.127209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:38.127209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:38.127209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:38.127209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:38.127209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:38.127209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:38.127209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:38.129210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:38.130113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:38.130113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:38.130113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:38.130113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:38.130113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:38.130113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:38.130113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:38.130113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:38.130113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 411/807] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:38.130954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be280f52-fe8a-4a00-ba01-abba60d59748 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:38.154303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:38.214399 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcca794-51b5-417f-8309-a9d90d9aebaf None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:38.224194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:38.243789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be280f52-fe8a-4a00-ba01-abba60d59748 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:38.258491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84890f03-fdd8-4963-9ed7-dcd3298d4cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:38.259149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84890f03-fdd8-4963-9ed7-dcd3298d4cd3 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:38.259428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84890f03-fdd8-4963-9ed7-dcd3298d4cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:38.259698 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84890f03-fdd8-4963-9ed7-dcd3298d4cd3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:38.260437 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84890f03-fdd8-4963-9ed7-dcd3298d4cd3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:38.261829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84890f03-fdd8-4963-9ed7-dcd3298d4cd3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:38.285164 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 397/808] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Jul 14 10:01:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:01:38.297687 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a9e20-2a0e-46aa-a85c-d8ba78171b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:38.298510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a9e20-2a0e-46aa-a85c-d8ba78171b55 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:38.298788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a9e20-2a0e-46aa-a85c-d8ba78171b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:38.299049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a9e20-2a0e-46aa-a85c-d8ba78171b55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:38.301161 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84890f03-fdd8-4963-9ed7-dcd3298d4cd3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:38.317799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a9e20-2a0e-46aa-a85c-d8ba78171b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:38.320576 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a9e20-2a0e-46aa-a85c-d8ba78171b55 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:38.347239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-246a9e20-2a0e-46aa-a85c-d8ba78171b55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:38.347239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:38.347239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:38.347239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:38.347239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:38.347239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:38.347239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:38.347239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:38.347239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:38.347239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:38.347239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:38.347239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:38.347239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:38.347239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:38.347239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:38.348238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:38.349231 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:38.349231 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:38.349231 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:38.349231 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:38.352043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 412/809] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:38.353722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a9e20-2a0e-46aa-a85c-d8ba78171b55 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:38.417932 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:38.425483 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84890f03-fdd8-4963-9ed7-dcd3298d4cd3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:38.465424 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee2b0d-8412-4035-b16c-91d456738c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:38.466705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee2b0d-8412-4035-b16c-91d456738c4a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:38.467144 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee2b0d-8412-4035-b16c-91d456738c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:38.467608 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee2b0d-8412-4035-b16c-91d456738c4a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:38.480019 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee2b0d-8412-4035-b16c-91d456738c4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:38.486899 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a9e20-2a0e-46aa-a85c-d8ba78171b55 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-6dee2b0d-8412-4035-b16c-91d456738c4a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:38.489383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:38.490381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:38.491211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:38.491211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:38.491211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:38.491211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:38.491211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:38.491211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:38.491211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:38.491211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:38.491211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:38.491211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:38.491211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:38.491211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:38.491211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 398/810] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:38.491897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee2b0d-8412-4035-b16c-91d456738c4a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:38.536465 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa9265-0215-41ec-b40f-7c313b2571d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:38.537178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa9265-0215-41ec-b40f-7c313b2571d5 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:38.537439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa9265-0215-41ec-b40f-7c313b2571d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:38.537712 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa9265-0215-41ec-b40f-7c313b2571d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:38.538540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa9265-0215-41ec-b40f-7c313b2571d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:38.538822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa9265-0215-41ec-b40f-7c313b2571d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:38.540248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa9265-0215-41ec-b40f-7c313b2571d5 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:38.541930 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:38.560010 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 413/811] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:38.561206 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa9265-0215-41ec-b40f-7c313b2571d5 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:38.616164 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee2b0d-8412-4035-b16c-91d456738c4a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:38.651139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad5afd-960f-4eb9-a352-fbf28159107c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:38.651800 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad5afd-960f-4eb9-a352-fbf28159107c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:38.652081 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad5afd-960f-4eb9-a352-fbf28159107c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:38.652359 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad5afd-960f-4eb9-a352-fbf28159107c None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:38.653244 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad5afd-960f-4eb9-a352-fbf28159107c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:38.653552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad5afd-960f-4eb9-a352-fbf28159107c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:38.655003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad5afd-960f-4eb9-a352-fbf28159107c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:38.666631 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 399/812] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:38.667917 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170cdc3-7ed7-493b-86cb-54422a5a247f None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:38.668257 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170cdc3-7ed7-493b-86cb-54422a5a247f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:38.668476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7170cdc3-7ed7-493b-86cb-54422a5a247f None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:38.668624 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa9265-0215-41ec-b40f-7c313b2571d5 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:38.670027 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 400/813] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:38.670716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170cdc3-7ed7-493b-86cb-54422a5a247f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:38.691735 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:38.712139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2a0e9-2c8b-4438-b5c2-c51b1b56b89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:38.713117 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2a0e9-2c8b-4438-b5c2-c51b1b56b89f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:38.713605 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2a0e9-2c8b-4438-b5c2-c51b1b56b89f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:38.714067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a2a0e9-2c8b-4438-b5c2-c51b1b56b89f None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:38.715006 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2a0e9-2c8b-4438-b5c2-c51b1b56b89f None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:38.715489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2a0e9-2c8b-4438-b5c2-c51b1b56b89f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:38.717172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a2a0e9-2c8b-4438-b5c2-c51b1b56b89f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:38.726763 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 414/814] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:38.727842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2a0e9-2c8b-4438-b5c2-c51b1b56b89f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:38.779243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7170cdc3-7ed7-493b-86cb-54422a5a247f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:38.787917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:38.814509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116a933-53da-48e6-917b-735e268a8d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:38.815450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116a933-53da-48e6-917b-735e268a8d9b None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:38.815727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116a933-53da-48e6-917b-735e268a8d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:38.816000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116a933-53da-48e6-917b-735e268a8d9b None admin] PATH_INFO: `/v3/projects/57662256880f43018039715a755c087c/users/d5fa502e8b014c719fcece0a130fdd7e/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:38.818029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116a933-53da-48e6-917b-735e268a8d9b None admin] RBAC: Authorizing `identity:create_grant(project_id=57662256880f43018039715a755c087c, user_id=d5fa502e8b014c719fcece0a130fdd7e, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:38.819468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7116a933-53da-48e6-917b-735e268a8d9b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:38.834538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 401/815] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:01:38 2022] PUT /identity/v3/projects/57662256880f43018039715a755c087c/users/d5fa502e8b014c719fcece0a130fdd7e/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:01:38.835776 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7ad74-2aca-4c3e-b641-f040050bfd49 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:38.836064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7ad74-2aca-4c3e-b641-f040050bfd49 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:38.836341 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7ad74-2aca-4c3e-b641-f040050bfd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:38.850794 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a2a0e9-2c8b-4438-b5c2-c51b1b56b89f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:38.859891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-17a7ad74-2aca-4c3e-b641-f040050bfd49 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:38.861157 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-17a7ad74-2aca-4c3e-b641-f040050bfd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:38.892183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a620e-527b-4203-99a6-79b13f341ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:38.894347 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a620e-527b-4203-99a6-79b13f341ba4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:38.894825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a620e-527b-4203-99a6-79b13f341ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:38.895098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a620e-527b-4203-99a6-79b13f341ba4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:38.903587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a620e-527b-4203-99a6-79b13f341ba4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:38.905413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a620e-527b-4203-99a6-79b13f341ba4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:38.922932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-022a620e-527b-4203-99a6-79b13f341ba4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:38.922932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:38.922932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:38.922932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:38.922932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:38.922932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:38.922932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:38.922932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:38.922932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:38.922932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:38.922932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:38.922932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:38.922932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:38.922932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:38.922932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:38.923888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:38.924869 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:38.924869 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:38.924869 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:38.924869 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:38.924869 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:38.924869 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:38.924869 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:38.924869 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:38.924869 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 415/816] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:38.925404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a620e-527b-4203-99a6-79b13f341ba4 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:38.952873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 402/817] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:38.954235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7ad74-2aca-4c3e-b641-f040050bfd49 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:38.964683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:39.070878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a620e-527b-4203-99a6-79b13f341ba4 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:39.089167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7ad74-2aca-4c3e-b641-f040050bfd49 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:39.091739 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:39.110993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100891fe-56ea-468f-867f-37233993d126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:39.111803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100891fe-56ea-468f-867f-37233993d126 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:39.112112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100891fe-56ea-468f-867f-37233993d126 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:39.112387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100891fe-56ea-468f-867f-37233993d126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:39.122971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100891fe-56ea-468f-867f-37233993d126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:39.124471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100891fe-56ea-468f-867f-37233993d126 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:39.131849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10842a-d36d-4aa0-8fc7-a3de0a021c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:39.133294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10842a-d36d-4aa0-8fc7-a3de0a021c81 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:39.133723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10842a-d36d-4aa0-8fc7-a3de0a021c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:39.134628 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10842a-d36d-4aa0-8fc7-a3de0a021c81 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:39.135850 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10842a-d36d-4aa0-8fc7-a3de0a021c81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:39.136310 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10842a-d36d-4aa0-8fc7-a3de0a021c81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:39.137920 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10842a-d36d-4aa0-8fc7-a3de0a021c81 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:39.143796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-100891fe-56ea-468f-867f-37233993d126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:39.143796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:39.143796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:39.143796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:39.143796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:39.143796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:39.143796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:39.143796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:39.143796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:39.143796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:39.143796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:39.143796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:39.143796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:39.143796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:39.143796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:39.144846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:39.146168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:39.146168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:39.146168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:39.146168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:39.146168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 416/818] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:39.146462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100891fe-56ea-468f-867f-37233993d126 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:39.155392 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 403/819] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:39.156525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10842a-d36d-4aa0-8fc7-a3de0a021c81 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:39.251371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:39.263898 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100891fe-56ea-468f-867f-37233993d126 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:39.284234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10842a-d36d-4aa0-8fc7-a3de0a021c81 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:39.300016 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa2cb0-e07f-4a7e-b332-eb85debe4933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:39.301115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa2cb0-e07f-4a7e-b332-eb85debe4933 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:39.301398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa2cb0-e07f-4a7e-b332-eb85debe4933 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:39.301673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fa2cb0-e07f-4a7e-b332-eb85debe4933 None admin] PATH_INFO: `/v3/services/network` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:39.302553 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa2cb0-e07f-4a7e-b332-eb85debe4933 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:39.304486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fa2cb0-e07f-4a7e-b332-eb85debe4933 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-d5fa2cb0-e07f-4a7e-b332-eb85debe4933 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:39.320570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:39.321615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:39.322591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:39.322591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:39.322591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:39.322591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:39.322591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:39.322591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:39.322591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:39.322591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:39.322591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:39.322591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:39.322591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:39.322591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:39.322591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 14 10:01:39.322591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:39.322591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 417/820] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:39 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:39.323382 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa2cb0-e07f-4a7e-b332-eb85debe4933 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:39.324735 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24446db8-ed8d-46cd-95c3-768b964fbaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:39.325770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24446db8-ed8d-46cd-95c3-768b964fbaa4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:39.326244 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24446db8-ed8d-46cd-95c3-768b964fbaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:39.326759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24446db8-ed8d-46cd-95c3-768b964fbaa4 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:39.327921 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24446db8-ed8d-46cd-95c3-768b964fbaa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:39.328426 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24446db8-ed8d-46cd-95c3-768b964fbaa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:39.330108 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24446db8-ed8d-46cd-95c3-768b964fbaa4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:39.347042 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 404/821] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:39.348190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24446db8-ed8d-46cd-95c3-768b964fbaa4 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:39.360339 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:39.447418 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fa2cb0-e07f-4a7e-b332-eb85debe4933 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:39.460884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24446db8-ed8d-46cd-95c3-768b964fbaa4 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:39.482324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41abe2-0668-4947-8890-13fb3b8d722c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:39.482967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41abe2-0668-4947-8890-13fb3b8d722c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:39.483251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41abe2-0668-4947-8890-13fb3b8d722c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:39.484115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41abe2-0668-4947-8890-13fb3b8d722c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:39.485175 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41abe2-0668-4947-8890-13fb3b8d722c None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=57662256880f43018039715a755c087c, user.id=e9f4efa3e2f343c2ac858a277601a7df) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:39.485610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41abe2-0668-4947-8890-13fb3b8d722c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:39.487342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a41abe2-0668-4947-8890-13fb3b8d722c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:39.490461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:39.497071 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 418/822] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:39 2022] GET /identity/v3/role_assignments?user.id=e9f4efa3e2f343c2ac858a277601a7df&scope.project.id=57662256880f43018039715a755c087c&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:39.498362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41abe2-0668-4947-8890-13fb3b8d722c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:39.511940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff80c2b8-bc72-42b0-9b05-d946e9564689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:39.516165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80c2b8-bc72-42b0-9b05-d946e9564689 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:39.516775 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80c2b8-bc72-42b0-9b05-d946e9564689 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:39.517280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80c2b8-bc72-42b0-9b05-d946e9564689 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:39.518822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff80c2b8-bc72-42b0-9b05-d946e9564689 None admin] RBAC: Adding query filter params (name=network) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:39.519345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff80c2b8-bc72-42b0-9b05-d946e9564689 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:39.521987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff80c2b8-bc72-42b0-9b05-d946e9564689 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:39.533004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 405/823] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Jul 14 10:01:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:39.539007 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff80c2b8-bc72-42b0-9b05-d946e9564689 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:39.662913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff80c2b8-bc72-42b0-9b05-d946e9564689 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:39.708664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f884f-9016-4354-8a71-7716e85bfde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:39.708664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f884f-9016-4354-8a71-7716e85bfde4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:39.708664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f884f-9016-4354-8a71-7716e85bfde4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:39.708664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f884f-9016-4354-8a71-7716e85bfde4 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:39.709127 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:39.709206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f884f-9016-4354-8a71-7716e85bfde4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:39.709206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f884f-9016-4354-8a71-7716e85bfde4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:39.709206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f884f-9016-4354-8a71-7716e85bfde4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:39.711909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41abe2-0668-4947-8890-13fb3b8d722c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:39.713381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 406/824] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Jul 14 10:01:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:39.720013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f884f-9016-4354-8a71-7716e85bfde4 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:39.759585 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f02f3-5e3f-4c3d-ad87-a880ffafe017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:39.760572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f02f3-5e3f-4c3d-ad87-a880ffafe017 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:39.760848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f02f3-5e3f-4c3d-ad87-a880ffafe017 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:39.761185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f02f3-5e3f-4c3d-ad87-a880ffafe017 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:39.772248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f02f3-5e3f-4c3d-ad87-a880ffafe017 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:39.773919 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8f02f3-5e3f-4c3d-ad87-a880ffafe017 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-9f8f02f3-5e3f-4c3d-ad87-a880ffafe017 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:39.792671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:39.793702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:39.794654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:39.794654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:39.794654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:39.794654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:39.794654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:39.794654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:39.794654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:39.794654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:39.794654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:01:39.794654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:39.794654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 419/825] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Jul 14 10:01:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:39.800852 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f02f3-5e3f-4c3d-ad87-a880ffafe017 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:39.806554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:39.860725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f884f-9016-4354-8a71-7716e85bfde4 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:39.902947 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf68cd3-a8cb-4170-bdfe-558d72c9b1f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:39.903957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf68cd3-a8cb-4170-bdfe-558d72c9b1f8 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:39.904467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf68cd3-a8cb-4170-bdfe-558d72c9b1f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:39.904960 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf68cd3-a8cb-4170-bdfe-558d72c9b1f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:39.905907 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf68cd3-a8cb-4170-bdfe-558d72c9b1f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:39.907469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf68cd3-a8cb-4170-bdfe-558d72c9b1f8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:39.928125 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8f02f3-5e3f-4c3d-ad87-a880ffafe017 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:39.934096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 407/826] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Jul 14 10:01:39 2022] POST /identity/v3/endpoints => generated 327 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:01:39.981946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4dac1d-60f5-4d12-8935-a3d886df2d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:39.982632 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dac1d-60f5-4d12-8935-a3d886df2d0d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:39.982914 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dac1d-60f5-4d12-8935-a3d886df2d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:39.983248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4dac1d-60f5-4d12-8935-a3d886df2d0d None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:39.984232 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4dac1d-60f5-4d12-8935-a3d886df2d0d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:39.984549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4dac1d-60f5-4d12-8935-a3d886df2d0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:39.986367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4dac1d-60f5-4d12-8935-a3d886df2d0d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:40.009982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 420/827] 10.222.0.61 () {56 vars in 1149 bytes} [Thu Jul 14 10:01:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:40.018132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf68cd3-a8cb-4170-bdfe-558d72c9b1f8 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:40.140385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf68cd3-a8cb-4170-bdfe-558d72c9b1f8 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:40.153101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:40.182209 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0b783-b07d-449b-9dda-430c6dc8a396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:40.183007 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0b783-b07d-449b-9dda-430c6dc8a396 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:40.183288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0b783-b07d-449b-9dda-430c6dc8a396 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:40.183566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0b783-b07d-449b-9dda-430c6dc8a396 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:40.195008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0b783-b07d-449b-9dda-430c6dc8a396 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:40.196457 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d0b783-b07d-449b-9dda-430c6dc8a396 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:40.215003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-79d0b783-b07d-449b-9dda-430c6dc8a396 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:40.215003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:40.215003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:40.215003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:40.215003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:40.215003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:40.215003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:40.215003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:40.215003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:40.215003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:40.215003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:40.215003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:40.215003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:40.215003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:40.215003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:40.215950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:40.216854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:40.216854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:40.216854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:40.216854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:40.216854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 408/828] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:40.222660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4dac1d-60f5-4d12-8935-a3d886df2d0d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:40.237567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:40.338571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4dac1d-60f5-4d12-8935-a3d886df2d0d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:40.377524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a159a6d-fac7-471b-b86c-21fccf1517eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:40.378192 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a159a6d-fac7-471b-b86c-21fccf1517eb None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:40.378470 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a159a6d-fac7-471b-b86c-21fccf1517eb None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:40.378746 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a159a6d-fac7-471b-b86c-21fccf1517eb None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:40.379474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a159a6d-fac7-471b-b86c-21fccf1517eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:40.379778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a159a6d-fac7-471b-b86c-21fccf1517eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:40.381174 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a159a6d-fac7-471b-b86c-21fccf1517eb None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:40.394083 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 421/829] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:40.400681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0b783-b07d-449b-9dda-430c6dc8a396 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:40.508303 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fed153-f741-4970-b44e-4967be505898 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:40.508591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fed153-f741-4970-b44e-4967be505898 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:40.508870 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fed153-f741-4970-b44e-4967be505898 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:40.510384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 422/830] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:40.515740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d0b783-b07d-449b-9dda-430c6dc8a396 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:40.518848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d65d02-6ea4-46b5-9ab6-410d87e9a00d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:40.519132 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d65d02-6ea4-46b5-9ab6-410d87e9a00d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:40.519405 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d65d02-6ea4-46b5-9ab6-410d87e9a00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:40.540955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c4d65d02-6ea4-46b5-9ab6-410d87e9a00d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:40.542779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c4d65d02-6ea4-46b5-9ab6-410d87e9a00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:40.555530 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760a41c-2e55-4bae-989f-355133c25f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:40.556443 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760a41c-2e55-4bae-989f-355133c25f41 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:40.556731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760a41c-2e55-4bae-989f-355133c25f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:40.557038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760a41c-2e55-4bae-989f-355133c25f41 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:40.564481 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760a41c-2e55-4bae-989f-355133c25f41 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:40.566119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760a41c-2e55-4bae-989f-355133c25f41 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:40.583082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-7760a41c-2e55-4bae-989f-355133c25f41 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:40.583082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:40.583082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:40.583082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:40.583082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:40.583082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:40.583082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:40.583082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:40.583082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:40.583082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:40.583082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:40.583082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:40.583082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:40.583082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:40.583082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:40.584014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:40.584979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:40.584979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:40.584979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:40.584979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:40.584979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:40.584979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:40.584979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:40.584979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:40.584979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 409/831] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:40.591920 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760a41c-2e55-4bae-989f-355133c25f41 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:40.618992 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:40.621883 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:40.632946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 423/832] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:01:40.654967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d65d02-6ea4-46b5-9ab6-410d87e9a00d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:40.708868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760a41c-2e55-4bae-989f-355133c25f41 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:40.748200 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1113de-f8e6-49d3-b94b-a6f01bf5cfcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:40.749145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1113de-f8e6-49d3-b94b-a6f01bf5cfcd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:40.749523 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1113de-f8e6-49d3-b94b-a6f01bf5cfcd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:40.749893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1113de-f8e6-49d3-b94b-a6f01bf5cfcd None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:40.750783 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1113de-f8e6-49d3-b94b-a6f01bf5cfcd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:40.751310 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1113de-f8e6-49d3-b94b-a6f01bf5cfcd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:40.753512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1113de-f8e6-49d3-b94b-a6f01bf5cfcd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:40.768911 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d65d02-6ea4-46b5-9ab6-410d87e9a00d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:40.772107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 410/833] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:40.773101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1113de-f8e6-49d3-b94b-a6f01bf5cfcd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:40.806081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26607a-bfcb-4456-93bb-3e59d7398651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:40.807011 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26607a-bfcb-4456-93bb-3e59d7398651 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:40.807292 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26607a-bfcb-4456-93bb-3e59d7398651 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:40.807571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26607a-bfcb-4456-93bb-3e59d7398651 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:40.808222 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26607a-bfcb-4456-93bb-3e59d7398651 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:40.809618 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26607a-bfcb-4456-93bb-3e59d7398651 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-0f26607a-bfcb-4456-93bb-3e59d7398651 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:40.822780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:40.823806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:40.824656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:40.824656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:40.824656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:40.824656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:40.824656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:40.824656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:40.824656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:40.824656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:40.824656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:40.824656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:40.824656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:40.824656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:40.824656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:01:40.824656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:40.824656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 424/834] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:01:40 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:40.832412 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26607a-bfcb-4456-93bb-3e59d7398651 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:40.874443 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:40.897943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1113de-f8e6-49d3-b94b-a6f01bf5cfcd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:40.916305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:40.941218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699775c-6918-4100-919e-b1bc9cb091b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:40.942115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699775c-6918-4100-919e-b1bc9cb091b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:40.942394 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699775c-6918-4100-919e-b1bc9cb091b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:40.942669 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1699775c-6918-4100-919e-b1bc9cb091b2 None admin] PATH_INFO: `/v3/projects/57662256880f43018039715a755c087c/users/dd2b9ad7efa848938627385b411727ee/roles/ba4088a289ea4b9a9aa9d9cd7a27e731` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:40.961708 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699775c-6918-4100-919e-b1bc9cb091b2 None admin] RBAC: Authorizing `identity:create_grant(project_id=57662256880f43018039715a755c087c, user_id=dd2b9ad7efa848938627385b411727ee, role_id=ba4088a289ea4b9a9aa9d9cd7a27e731)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:40.963313 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1699775c-6918-4100-919e-b1bc9cb091b2 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:40.967577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26607a-bfcb-4456-93bb-3e59d7398651 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:40.987320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 411/835] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:01:40 2022] PUT /identity/v3/projects/57662256880f43018039715a755c087c/users/dd2b9ad7efa848938627385b411727ee/roles/ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:01:41.019630 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789341d8-11cf-4951-81fe-24937ad1a01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:41.020313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789341d8-11cf-4951-81fe-24937ad1a01b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:41.020594 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789341d8-11cf-4951-81fe-24937ad1a01b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:41.020934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789341d8-11cf-4951-81fe-24937ad1a01b None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:41.021778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789341d8-11cf-4951-81fe-24937ad1a01b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:41.022134 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789341d8-11cf-4951-81fe-24937ad1a01b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:41.023539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789341d8-11cf-4951-81fe-24937ad1a01b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:41.029602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 425/836] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:01:40 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:41.036618 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699775c-6918-4100-919e-b1bc9cb091b2 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:41.070429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3db2f-d180-4b7c-b3d3-4a3f52ae4529 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:41.070740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3db2f-d180-4b7c-b3d3-4a3f52ae4529 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:41.071018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3db2f-d180-4b7c-b3d3-4a3f52ae4529 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:41.072566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 426/837] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:41.078742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee969495-1875-4a59-987b-ef411f84c8e1 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:41.079352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee969495-1875-4a59-987b-ef411f84c8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:41.079850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee969495-1875-4a59-987b-ef411f84c8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:41.100791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ee969495-1875-4a59-987b-ef411f84c8e1 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:41.102497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ee969495-1875-4a59-987b-ef411f84c8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:41.171331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1699775c-6918-4100-919e-b1bc9cb091b2 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:41.183099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 427/838] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:01:41.208918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee969495-1875-4a59-987b-ef411f84c8e1 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:41.211758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:41.217119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1106fc-be56-46d1-b13a-2c5e1f172786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:41.217957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1106fc-be56-46d1-b13a-2c5e1f172786 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:41.218417 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1106fc-be56-46d1-b13a-2c5e1f172786 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:41.218843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1106fc-be56-46d1-b13a-2c5e1f172786 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:41.219893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1106fc-be56-46d1-b13a-2c5e1f172786 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:41.220489 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1106fc-be56-46d1-b13a-2c5e1f172786 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:41.222086 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1106fc-be56-46d1-b13a-2c5e1f172786 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:41.228334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 412/839] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:01:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:41.285732 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:41.334520 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee969495-1875-4a59-987b-ef411f84c8e1 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:41.371449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c06c2-21aa-4309-9121-70c6a69547dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:41.372258 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c06c2-21aa-4309-9121-70c6a69547dd None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:41.372540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c06c2-21aa-4309-9121-70c6a69547dd None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:41.372814 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c06c2-21aa-4309-9121-70c6a69547dd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:41.385253 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045c06c2-21aa-4309-9121-70c6a69547dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-045c06c2-21aa-4309-9121-70c6a69547dd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:41.393068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:41.394046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:41.394937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:41.394937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:41.394937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:41.394937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:41.394937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:41.394937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:41.394937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:41.394937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:41.394937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:41.394937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:41.394937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:01:41.394937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:41.394937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 428/840] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:01:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:41.400145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1106fc-be56-46d1-b13a-2c5e1f172786 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:41.507406 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1106fc-be56-46d1-b13a-2c5e1f172786 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:41.542967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c929d96-7b43-4748-aefe-d43ccfbc042b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:41.543636 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c929d96-7b43-4748-aefe-d43ccfbc042b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:41.543911 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c929d96-7b43-4748-aefe-d43ccfbc042b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:41.544178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c929d96-7b43-4748-aefe-d43ccfbc042b None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:41.544937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c929d96-7b43-4748-aefe-d43ccfbc042b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:41.545247 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c929d96-7b43-4748-aefe-d43ccfbc042b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:41.546606 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c929d96-7b43-4748-aefe-d43ccfbc042b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:41.556720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 413/841] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:01:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:41.561974 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c06c2-21aa-4309-9121-70c6a69547dd None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:41.667346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c06c2-21aa-4309-9121-70c6a69547dd None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:41.685319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:41.702372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c5e72-de5f-4025-8ed4-788043393257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:41.703175 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c5e72-de5f-4025-8ed4-788043393257 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:41.703455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c5e72-de5f-4025-8ed4-788043393257 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:41.703727 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c5e72-de5f-4025-8ed4-788043393257 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:41.715152 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c5e72-de5f-4025-8ed4-788043393257 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:41.716605 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c5e72-de5f-4025-8ed4-788043393257 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:41.734724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-327c5e72-de5f-4025-8ed4-788043393257 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:41.734724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:41.734724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:41.734724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:41.734724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:41.734724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:41.734724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:41.734724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:41.734724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:41.734724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:41.734724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:41.734724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:41.734724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:41.734724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:41.734724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:41.735738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:41.737443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:41.737443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:41.737443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:41.737443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:41.737443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 429/842] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:41.742151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c929d96-7b43-4748-aefe-d43ccfbc042b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:41.804208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:41.851711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c929d96-7b43-4748-aefe-d43ccfbc042b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:41.888738 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455b97f-4824-4818-85a3-7f25cf996895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:41.890530 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455b97f-4824-4818-85a3-7f25cf996895 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:41.891031 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455b97f-4824-4818-85a3-7f25cf996895 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:41.891503 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d455b97f-4824-4818-85a3-7f25cf996895 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:41.892428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455b97f-4824-4818-85a3-7f25cf996895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:41.892997 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455b97f-4824-4818-85a3-7f25cf996895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:41.894646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d455b97f-4824-4818-85a3-7f25cf996895 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:41.908113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 414/843] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:41.917050 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c5e72-de5f-4025-8ed4-788043393257 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:42.017550 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c5e72-de5f-4025-8ed4-788043393257 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:42.050875 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23efa46-e5b2-4f45-a97d-e94a9a2f50da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:42.051665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23efa46-e5b2-4f45-a97d-e94a9a2f50da None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:42.051943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23efa46-e5b2-4f45-a97d-e94a9a2f50da None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:42.052214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23efa46-e5b2-4f45-a97d-e94a9a2f50da None admin] PATH_INFO: `/v3/users/swift` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:42.060226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23efa46-e5b2-4f45-a97d-e94a9a2f50da None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:42.061622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23efa46-e5b2-4f45-a97d-e94a9a2f50da None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-e23efa46-e5b2-4f45-a97d-e94a9a2f50da None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:42.076235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:42.077270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:42.078248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:42.078248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:42.078248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:42.078248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:42.078248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:42.078248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:42.078248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:42.078248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:42.078248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 14 10:01:42.078248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:42.078248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 430/844] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:01:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:42.083925 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455b97f-4824-4818-85a3-7f25cf996895 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:42.110488 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245d6aa-1d9a-46e6-87f8-cc1e12a70585 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:42.110796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245d6aa-1d9a-46e6-87f8-cc1e12a70585 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:42.111330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9245d6aa-1d9a-46e6-87f8-cc1e12a70585 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:42.112653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 431/845] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:01:42.117710 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517b03e-85a9-4376-8613-2f50420d50d7 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:42.118099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517b03e-85a9-4376-8613-2f50420d50d7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:42.118447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517b03e-85a9-4376-8613-2f50420d50d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:42.135005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8517b03e-85a9-4376-8613-2f50420d50d7 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:42.136174 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8517b03e-85a9-4376-8613-2f50420d50d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:42.161707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:42.200954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d455b97f-4824-4818-85a3-7f25cf996895 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:42.225100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 432/846] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:42 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:01:42.244624 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517b03e-85a9-4376-8613-2f50420d50d7 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:42.249648 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:42.262035 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87523729-9747-4946-a0bc-536da3e72db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:42.262991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87523729-9747-4946-a0bc-536da3e72db9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:42.263487 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87523729-9747-4946-a0bc-536da3e72db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:42.264020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87523729-9747-4946-a0bc-536da3e72db9 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:42.265134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87523729-9747-4946-a0bc-536da3e72db9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:42.265636 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87523729-9747-4946-a0bc-536da3e72db9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:42.267297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87523729-9747-4946-a0bc-536da3e72db9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:42.290840 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 415/847] 10.222.0.61 () {56 vars in 1135 bytes} [Thu Jul 14 10:01:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:42.292009 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87523729-9747-4946-a0bc-536da3e72db9 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:42.344354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517b03e-85a9-4376-8613-2f50420d50d7 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:42.379783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89394bbc-20b2-4cd1-9674-407b783af068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:42.380590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89394bbc-20b2-4cd1-9674-407b783af068 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:42.380868 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89394bbc-20b2-4cd1-9674-407b783af068 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:42.381181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89394bbc-20b2-4cd1-9674-407b783af068 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:42.382017 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89394bbc-20b2-4cd1-9674-407b783af068 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:42.383192 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89394bbc-20b2-4cd1-9674-407b783af068 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-89394bbc-20b2-4cd1-9674-407b783af068 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:42.395436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:42.396369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:42.397241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:42.397241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:42.397241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:42.397241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:42.397241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:42.397241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:42.397241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:42.397241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:42.397241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:42.397241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:42.397241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:42.397241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:42.397241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:42.397241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:01:42.397241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:42.401474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 433/848] 10.222.0.61 () {56 vars in 1132 bytes} [Thu Jul 14 10:01:42 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:42.402655 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89394bbc-20b2-4cd1-9674-407b783af068 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:42.403057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87523729-9747-4946-a0bc-536da3e72db9 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:42.429684 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:42.441384 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ba57b-1c19-4a93-a4e7-007ec3647a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:42.442345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ba57b-1c19-4a93-a4e7-007ec3647a4f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:42.442825 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ba57b-1c19-4a93-a4e7-007ec3647a4f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:42.443290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311ba57b-1c19-4a93-a4e7-007ec3647a4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:42.491909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:42.510110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89394bbc-20b2-4cd1-9674-407b783af068 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:42.521292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ba57b-1c19-4a93-a4e7-007ec3647a4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:42.522961 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311ba57b-1c19-4a93-a4e7-007ec3647a4f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:42.545388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be54556-d5ed-4a33-add6-a9943ea46139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:42.546055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be54556-d5ed-4a33-add6-a9943ea46139 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:42.546336 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be54556-d5ed-4a33-add6-a9943ea46139 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:42.546609 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be54556-d5ed-4a33-add6-a9943ea46139 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:42.547309 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be54556-d5ed-4a33-add6-a9943ea46139 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:42.547626 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be54556-d5ed-4a33-add6-a9943ea46139 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:42.549574 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be54556-d5ed-4a33-add6-a9943ea46139 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:42.559079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 434/849] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Jul 14 10:01:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:42.560255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be54556-d5ed-4a33-add6-a9943ea46139 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:42.634616 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 416/850] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:42 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:01:42.635849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ba57b-1c19-4a93-a4e7-007ec3647a4f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:42.652816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:42.666197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be54556-d5ed-4a33-add6-a9943ea46139 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:42.701398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58967af-b296-400e-8e66-e91039d808f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:42.702114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58967af-b296-400e-8e66-e91039d808f7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:42.702411 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58967af-b296-400e-8e66-e91039d808f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:42.702669 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58967af-b296-400e-8e66-e91039d808f7 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:42.703387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58967af-b296-400e-8e66-e91039d808f7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:42.703693 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58967af-b296-400e-8e66-e91039d808f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:42.705596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58967af-b296-400e-8e66-e91039d808f7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:42.717200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 435/851] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Jul 14 10:01:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:42.719104 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:42.748224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311ba57b-1c19-4a93-a4e7-007ec3647a4f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:42.788741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38730c-9bfa-4997-be92-20006e62a9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:42.789975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38730c-9bfa-4997-be92-20006e62a9ea None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:42.790544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38730c-9bfa-4997-be92-20006e62a9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:42.791062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f38730c-9bfa-4997-be92-20006e62a9ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:42.803715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f38730c-9bfa-4997-be92-20006e62a9ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:42.805594 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f38730c-9bfa-4997-be92-20006e62a9ea None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:42.825235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-8f38730c-9bfa-4997-be92-20006e62a9ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:42.825235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:42.825235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:42.825235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:42.825235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:42.825235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:42.825235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:42.825235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:42.825235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:42.825235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:42.825235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:42.825235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:42.825235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:42.825235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:42.825235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:42.826208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:42.827175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:42.827175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:42.827175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:42.827175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:42.829557 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 417/852] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:42.836049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58967af-b296-400e-8e66-e91039d808f7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:42.942747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58967af-b296-400e-8e66-e91039d808f7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:42.977487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78f28e-5e16-4f48-9e5f-bd4e83958efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:42.978485 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78f28e-5e16-4f48-9e5f-bd4e83958efe None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:42.978960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78f28e-5e16-4f48-9e5f-bd4e83958efe None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:42.979447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78f28e-5e16-4f48-9e5f-bd4e83958efe None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:42.980373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78f28e-5e16-4f48-9e5f-bd4e83958efe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:42.980870 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78f28e-5e16-4f48-9e5f-bd4e83958efe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:42.982491 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78f28e-5e16-4f48-9e5f-bd4e83958efe None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:42.994894 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 436/853] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:43.000717 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38730c-9bfa-4997-be92-20006e62a9ea None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:43.108382 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f38730c-9bfa-4997-be92-20006e62a9ea None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:43.136335 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:43.145480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131ebd2-b29f-48d3-b5c3-733a075758d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:43.146580 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131ebd2-b29f-48d3-b5c3-733a075758d0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:43.147059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131ebd2-b29f-48d3-b5c3-733a075758d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:43.147540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4131ebd2-b29f-48d3-b5c3-733a075758d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:43.154757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4131ebd2-b29f-48d3-b5c3-733a075758d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:43.156434 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4131ebd2-b29f-48d3-b5c3-733a075758d0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:43.170859 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c362162-fb4c-4654-9aad-2b370edd033c None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:43.171353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c362162-fb4c-4654-9aad-2b370edd033c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:43.171456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c362162-fb4c-4654-9aad-2b370edd033c None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:43.172398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-4131ebd2-b29f-48d3-b5c3-733a075758d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:43.172398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:43.172398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:43.172398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:43.172398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:43.172398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:43.172398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:43.172398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:43.172398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:43.172398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:43.172398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:43.172398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:43.172398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:43.172398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:43.172398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:43.173259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:43.174739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:43.174739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:43.174739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:43.174739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:43.174739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:43.174739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:43.174739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:43.174739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:43.181335 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 437/854] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:43.181460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 418/855] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:43.182873 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e40165-e846-4f2c-9336-651395a142c0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:43.183219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e40165-e846-4f2c-9336-651395a142c0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:43.183600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e40165-e846-4f2c-9336-651395a142c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:43.186965 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131ebd2-b29f-48d3-b5c3-733a075758d0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:43.203347 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c5e40165-e846-4f2c-9336-651395a142c0 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:43.204603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c5e40165-e846-4f2c-9336-651395a142c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:43.218006 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:43.291041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 438/856] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:01:43.310257 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4131ebd2-b29f-48d3-b5c3-733a075758d0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:43.314782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e40165-e846-4f2c-9336-651395a142c0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:43.356305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c281f4-c246-4406-8e6b-9ec73a6a2b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:43.357428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c281f4-c246-4406-8e6b-9ec73a6a2b7d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:43.357938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c281f4-c246-4406-8e6b-9ec73a6a2b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:43.358411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c281f4-c246-4406-8e6b-9ec73a6a2b7d None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:43.359420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c281f4-c246-4406-8e6b-9ec73a6a2b7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:43.359928 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c281f4-c246-4406-8e6b-9ec73a6a2b7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:43.361661 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c281f4-c246-4406-8e6b-9ec73a6a2b7d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:43.375310 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 419/857] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:43.382190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c281f4-c246-4406-8e6b-9ec73a6a2b7d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:43.439362 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:43.442191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e40165-e846-4f2c-9336-651395a142c0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:43.485504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7b2968-189e-4a20-a612-b6416cb5b5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:43.486232 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b2968-189e-4a20-a612-b6416cb5b5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:43.486510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b2968-189e-4a20-a612-b6416cb5b5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:43.486777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7b2968-189e-4a20-a612-b6416cb5b5c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:43.497167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7b2968-189e-4a20-a612-b6416cb5b5c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:43.497389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c281f4-c246-4406-8e6b-9ec73a6a2b7d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-7e7b2968-189e-4a20-a612-b6416cb5b5c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:43.504828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:43.505815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:43.506677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:43.506677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:43.506677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:43.506677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:43.506677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:43.506677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:43.506677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:43.506677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:43.506677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:43.506677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:43.506677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:01:43.506677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:43.506677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 439/858] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:01:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:43.507862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d0fde-ec9e-4579-9a16-b1b5c44c919b None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:43.507862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d0fde-ec9e-4579-9a16-b1b5c44c919b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:43.508119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d0fde-ec9e-4579-9a16-b1b5c44c919b None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:43.513049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 440/859] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:43.513049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d0fde-ec9e-4579-9a16-b1b5c44c919b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:43.515479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:43.540355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975468e-c416-414d-b433-8d6102768e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:43.541033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975468e-c416-414d-b433-8d6102768e04 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:43.541315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975468e-c416-414d-b433-8d6102768e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:43.541586 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975468e-c416-414d-b433-8d6102768e04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:43.542412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975468e-c416-414d-b433-8d6102768e04 None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=57662256880f43018039715a755c087c, user.id=d5fa502e8b014c719fcece0a130fdd7e) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:43.542720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975468e-c416-414d-b433-8d6102768e04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:43.544126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975468e-c416-414d-b433-8d6102768e04 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:43.560975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 420/860] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:43 2022] GET /identity/v3/role_assignments?user.id=d5fa502e8b014c719fcece0a130fdd7e&scope.project.id=57662256880f43018039715a755c087c&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:43.560975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d788f-369d-4ce5-8665-0ccd17556716 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:43.560975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d788f-369d-4ce5-8665-0ccd17556716 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:43.560975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77d788f-369d-4ce5-8665-0ccd17556716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:43.569621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d77d788f-369d-4ce5-8665-0ccd17556716 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:43.570857 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d77d788f-369d-4ce5-8665-0ccd17556716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:43.646001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 421/861] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:43.657283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d0fde-ec9e-4579-9a16-b1b5c44c919b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:43.673425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d788f-369d-4ce5-8665-0ccd17556716 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:43.702028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f4ef3-3879-4da7-a727-b54c33fff4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:43.702839 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f4ef3-3879-4da7-a727-b54c33fff4ad None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:43.703386 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f4ef3-3879-4da7-a727-b54c33fff4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:43.703672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095f4ef3-3879-4da7-a727-b54c33fff4ad None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:43.704438 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f4ef3-3879-4da7-a727-b54c33fff4ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:43.705358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f4ef3-3879-4da7-a727-b54c33fff4ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:43.708054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095f4ef3-3879-4da7-a727-b54c33fff4ad None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:43.719990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 441/862] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:01:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:43.725959 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f4ef3-3879-4da7-a727-b54c33fff4ad None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:43.729412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:43.797330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:43.806232 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77d788f-369d-4ce5-8665-0ccd17556716 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:43.845768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095f4ef3-3879-4da7-a727-b54c33fff4ad None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:43.849711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055ce32-ee30-4f0f-92cc-3de7640bc843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:43.850398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055ce32-ee30-4f0f-92cc-3de7640bc843 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:43.850678 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055ce32-ee30-4f0f-92cc-3de7640bc843 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:43.850954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4055ce32-ee30-4f0f-92cc-3de7640bc843 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:43.851696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4055ce32-ee30-4f0f-92cc-3de7640bc843 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:43.853093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4055ce32-ee30-4f0f-92cc-3de7640bc843 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:43.872196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 422/863] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Jul 14 10:01:43 2022] POST /identity/v3/services => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:01:43.893565 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18175f8a-f82f-4afe-8e73-8cc65e950281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:43.894387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18175f8a-f82f-4afe-8e73-8cc65e950281 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:43.894664 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18175f8a-f82f-4afe-8e73-8cc65e950281 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:43.894935 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18175f8a-f82f-4afe-8e73-8cc65e950281 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:43.908074 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18175f8a-f82f-4afe-8e73-8cc65e950281 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:43.909571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18175f8a-f82f-4afe-8e73-8cc65e950281 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:43.929362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-18175f8a-f82f-4afe-8e73-8cc65e950281 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:43.929362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:43.929362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:43.929362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:43.929362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:43.929362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:43.929362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:43.929362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:43.929362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:43.929362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:43.929362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:43.929362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:43.929362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:43.929362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:43.929362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:43.930364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:43.931374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:43.931374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:43.931374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:43.931374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:43.931374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 442/864] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:43.937839 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055ce32-ee30-4f0f-92cc-3de7640bc843 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:44.060630 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4055ce32-ee30-4f0f-92cc-3de7640bc843 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:44.062273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:44.101241 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75f3e2-e620-4d1d-876c-6c1bffb6920c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:44.101910 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75f3e2-e620-4d1d-876c-6c1bffb6920c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:44.102194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75f3e2-e620-4d1d-876c-6c1bffb6920c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:44.102467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b75f3e2-e620-4d1d-876c-6c1bffb6920c None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:44.103226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75f3e2-e620-4d1d-876c-6c1bffb6920c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:44.103505 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75f3e2-e620-4d1d-876c-6c1bffb6920c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:44.104875 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b75f3e2-e620-4d1d-876c-6c1bffb6920c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:44.117353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 423/865] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:44.122400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18175f8a-f82f-4afe-8e73-8cc65e950281 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:44.231638 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18175f8a-f82f-4afe-8e73-8cc65e950281 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:44.253329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:44.267545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8dacb-f81d-4377-890a-4cbd7070de3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:44.268361 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8dacb-f81d-4377-890a-4cbd7070de3b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:44.268637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8dacb-f81d-4377-890a-4cbd7070de3b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:44.268907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f8dacb-f81d-4377-890a-4cbd7070de3b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:44.277061 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8dacb-f81d-4377-890a-4cbd7070de3b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:44.278500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f8dacb-f81d-4377-890a-4cbd7070de3b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-f8f8dacb-f81d-4377-890a-4cbd7070de3b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:44.293413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:44.294420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:44.295401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:44.295401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:44.295401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:44.295401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:44.295401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:44.295401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:44.295401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:44.295401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:44.295401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:01:44.295401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:44.295401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 443/866] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Jul 14 10:01:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:44.301596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75f3e2-e620-4d1d-876c-6c1bffb6920c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:44.412436 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b75f3e2-e620-4d1d-876c-6c1bffb6920c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:44.450912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9ad32-33e8-4f53-88de-0c3b9fe7636c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:44.451960 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9ad32-33e8-4f53-88de-0c3b9fe7636c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:44.452455 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9ad32-33e8-4f53-88de-0c3b9fe7636c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:44.452958 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9ad32-33e8-4f53-88de-0c3b9fe7636c None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:44.454091 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9ad32-33e8-4f53-88de-0c3b9fe7636c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:44.454736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9ad32-33e8-4f53-88de-0c3b9fe7636c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:44.456377 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b9ad32-33e8-4f53-88de-0c3b9fe7636c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:44.475466 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 424/867] 10.222.0.61 () {56 vars in 1149 bytes} [Thu Jul 14 10:01:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:44.484420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8dacb-f81d-4377-890a-4cbd7070de3b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:44.583943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f8dacb-f81d-4377-890a-4cbd7070de3b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:44.618412 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405c961-1859-4766-a40a-d8b12d3611fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:44.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405c961-1859-4766-a40a-d8b12d3611fc None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:44.619346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405c961-1859-4766-a40a-d8b12d3611fc None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:44.619612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405c961-1859-4766-a40a-d8b12d3611fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:44.665570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:44.688588 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405c961-1859-4766-a40a-d8b12d3611fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:44.690136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405c961-1859-4766-a40a-d8b12d3611fc None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:44.791908 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 444/868] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:44 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:01:44.797223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9ad32-33e8-4f53-88de-0c3b9fe7636c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:44.826004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:44.879308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8695545-7e87-48e2-b796-dfba29f58d08 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:44.879616 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8695545-7e87-48e2-b796-dfba29f58d08 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:44.879852 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8695545-7e87-48e2-b796-dfba29f58d08 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:44.881264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 445/869] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:01:44.885990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88080dad-d84c-46ae-a35b-6c1b04de62ec None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:44.886274 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88080dad-d84c-46ae-a35b-6c1b04de62ec None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:44.886547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88080dad-d84c-46ae-a35b-6c1b04de62ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:44.904014 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-88080dad-d84c-46ae-a35b-6c1b04de62ec None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:44.905237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-88080dad-d84c-46ae-a35b-6c1b04de62ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:44.910082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9ad32-33e8-4f53-88de-0c3b9fe7636c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:44.956467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b756b27-9968-4872-b98d-0906e03fa766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:44.957690 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b756b27-9968-4872-b98d-0906e03fa766 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:44.958242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b756b27-9968-4872-b98d-0906e03fa766 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:44.958528 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b756b27-9968-4872-b98d-0906e03fa766 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:44.973148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b756b27-9968-4872-b98d-0906e03fa766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:44.974601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b756b27-9968-4872-b98d-0906e03fa766 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:44.983247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 446/870] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:44 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:01:44.998554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-6b756b27-9968-4872-b98d-0906e03fa766 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:44.998554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:44.998554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:44.998554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:44.998554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:44.998554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:44.998554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:44.998554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:44.998554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:44.998554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:44.998554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:44.998554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:44.998554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:44.998554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:44.998554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:44.999472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:45.000373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:45.000373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:45.000373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:45.000373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:45.000373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 425/871] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:45.005938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b756b27-9968-4872-b98d-0906e03fa766 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:45.006264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88080dad-d84c-46ae-a35b-6c1b04de62ec None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:45.008896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:45.082496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:45.115217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88080dad-d84c-46ae-a35b-6c1b04de62ec None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:45.118370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b756b27-9968-4872-b98d-0906e03fa766 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:45.155945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd05643-6c46-4e6a-9b13-21943e038c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:45.156619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd05643-6c46-4e6a-9b13-21943e038c28 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:45.156897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd05643-6c46-4e6a-9b13-21943e038c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:45.157987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd05643-6c46-4e6a-9b13-21943e038c28 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:45.158714 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd05643-6c46-4e6a-9b13-21943e038c28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:45.160074 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd05643-6c46-4e6a-9b13-21943e038c28 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:45.165586 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd94e2-1663-4965-993c-418b02d9f2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:45.167449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd94e2-1663-4965-993c-418b02d9f2ac None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:45.167449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd94e2-1663-4965-993c-418b02d9f2ac None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:45.167449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cd94e2-1663-4965-993c-418b02d9f2ac None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:45.168382 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd94e2-1663-4965-993c-418b02d9f2ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:45.168829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd94e2-1663-4965-993c-418b02d9f2ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:45.170662 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cd94e2-1663-4965-993c-418b02d9f2ac None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:45.176606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 447/872] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Jul 14 10:01:45 2022] POST /identity/v3/services => generated 269 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:01:45.192973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 426/873] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:45.193319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd05643-6c46-4e6a-9b13-21943e038c28 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:45.273554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:45.337890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd05643-6c46-4e6a-9b13-21943e038c28 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:45.374413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8d179-a061-45bd-9b83-0ea892690882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:45.375221 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8d179-a061-45bd-9b83-0ea892690882 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:45.375503 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8d179-a061-45bd-9b83-0ea892690882 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:45.375777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8d179-a061-45bd-9b83-0ea892690882 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:45.383662 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8d179-a061-45bd-9b83-0ea892690882 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:45.385134 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8d179-a061-45bd-9b83-0ea892690882 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:45.399960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-22b8d179-a061-45bd-9b83-0ea892690882 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:45.399960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:45.399960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:45.399960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:45.399960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:45.399960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:45.399960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:45.399960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:45.399960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:45.399960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:45.399960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:45.399960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:45.399960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:45.399960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:45.399960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:45.400878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:45.401784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:45.401784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:45.401784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:45.401784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:45.401784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:45.401784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:45.401784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:45.401784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:45.401784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 448/874] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:45.407245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd94e2-1663-4965-993c-418b02d9f2ac None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:45.512285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cd94e2-1663-4965-993c-418b02d9f2ac None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:45.544544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:45.550607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbff43-5e71-4163-ad7f-25a73b49b9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:45.551285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbff43-5e71-4163-ad7f-25a73b49b9d2 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:45.551565 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbff43-5e71-4163-ad7f-25a73b49b9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:45.551840 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbff43-5e71-4163-ad7f-25a73b49b9d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:45.552894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbff43-5e71-4163-ad7f-25a73b49b9d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:45.553233 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbff43-5e71-4163-ad7f-25a73b49b9d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:45.554714 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbff43-5e71-4163-ad7f-25a73b49b9d2 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:45.568418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 427/875] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:45.574153 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8d179-a061-45bd-9b83-0ea892690882 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:45.678317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8d179-a061-45bd-9b83-0ea892690882 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:45.685719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693e93c-31f9-4d4a-b5fb-c0e192920a75 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:45.686000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693e93c-31f9-4d4a-b5fb-c0e192920a75 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:45.686279 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693e93c-31f9-4d4a-b5fb-c0e192920a75 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:45.687890 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 428/876] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:45.693741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684da4eb-dd17-4b8f-80d8-3fb9905ee7bb None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:45.694029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684da4eb-dd17-4b8f-80d8-3fb9905ee7bb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:45.694603 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684da4eb-dd17-4b8f-80d8-3fb9905ee7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:45.697922 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:45.716744 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-684da4eb-dd17-4b8f-80d8-3fb9905ee7bb None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:45.717940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-684da4eb-dd17-4b8f-80d8-3fb9905ee7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:45.724455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6c898-75a4-4f5c-8c9b-ff8e293053f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:45.725405 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6c898-75a4-4f5c-8c9b-ff8e293053f6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:45.726418 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6c898-75a4-4f5c-8c9b-ff8e293053f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:45.726901 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c6c898-75a4-4f5c-8c9b-ff8e293053f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:45.728407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c6c898-75a4-4f5c-8c9b-ff8e293053f6 None admin] RBAC: Adding query filter params (role.id=ba4088a289ea4b9a9aa9d9cd7a27e731, scope.project.id=57662256880f43018039715a755c087c, user.id=dd2b9ad7efa848938627385b411727ee) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:45.728947 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c6c898-75a4-4f5c-8c9b-ff8e293053f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:45.731419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c6c898-75a4-4f5c-8c9b-ff8e293053f6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:45.744192 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 449/877] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:45 2022] GET /identity/v3/role_assignments?user.id=dd2b9ad7efa848938627385b411727ee&scope.project.id=57662256880f43018039715a755c087c&role.id=ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:45.792748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 429/878] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:01:45.816493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6c898-75a4-4f5c-8c9b-ff8e293053f6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:45.931187 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c6c898-75a4-4f5c-8c9b-ff8e293053f6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:45.967087 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabe65a-b72f-4706-b5ae-7db7f5b01838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:45.968155 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabe65a-b72f-4706-b5ae-7db7f5b01838 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:45.968578 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabe65a-b72f-4706-b5ae-7db7f5b01838 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:45.969038 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabe65a-b72f-4706-b5ae-7db7f5b01838 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:45.970370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabe65a-b72f-4706-b5ae-7db7f5b01838 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:45.971889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dabe65a-b72f-4706-b5ae-7db7f5b01838 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-6dabe65a-b72f-4706-b5ae-7db7f5b01838 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:45.984681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:45.986530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:45.987372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:45.987372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:45.987372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:45.987372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:45.987372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:45.987372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:45.987372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:45.987372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:45.987372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:45.987372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:45.987372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:45.987372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:45.987372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:01:45.987372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:45.988115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 450/879] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Jul 14 10:01:45 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:45.997963 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684da4eb-dd17-4b8f-80d8-3fb9905ee7bb None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:46.109472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684da4eb-dd17-4b8f-80d8-3fb9905ee7bb None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:46.134131 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:46.146418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476e7c4-bd10-4ba0-bb39-04d6eff29640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:46.147089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476e7c4-bd10-4ba0-bb39-04d6eff29640 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:46.147370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476e7c4-bd10-4ba0-bb39-04d6eff29640 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:46.147645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2476e7c4-bd10-4ba0-bb39-04d6eff29640 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:46.148354 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476e7c4-bd10-4ba0-bb39-04d6eff29640 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:46.148658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476e7c4-bd10-4ba0-bb39-04d6eff29640 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:46.150076 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2476e7c4-bd10-4ba0-bb39-04d6eff29640 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:46.155041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 430/880] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Jul 14 10:01:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:46.161354 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476e7c4-bd10-4ba0-bb39-04d6eff29640 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:46.161788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ea603-f0a6-421d-828b-db9186e7340d None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:46.162078 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ea603-f0a6-421d-828b-db9186e7340d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:46.162355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ea603-f0a6-421d-828b-db9186e7340d None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:46.164163 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 451/881] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:46.168824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc1514-995d-4cc6-b9ef-499a2c92a032 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:46.169137 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc1514-995d-4cc6-b9ef-499a2c92a032 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:46.169414 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cc1514-995d-4cc6-b9ef-499a2c92a032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:46.188213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e6cc1514-995d-4cc6-b9ef-499a2c92a032 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:46.190610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e6cc1514-995d-4cc6-b9ef-499a2c92a032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:46.254894 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:46.277290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2476e7c4-bd10-4ba0-bb39-04d6eff29640 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:46.293014 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 452/882] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:46.307848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc1514-995d-4cc6-b9ef-499a2c92a032 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:46.314641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b09fec-becf-4e6e-8164-54ef8c80c7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:46.315685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b09fec-becf-4e6e-8164-54ef8c80c7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:46.316160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b09fec-becf-4e6e-8164-54ef8c80c7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:46.316761 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b09fec-becf-4e6e-8164-54ef8c80c7f3 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:46.317603 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b09fec-becf-4e6e-8164-54ef8c80c7f3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:46.318006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b09fec-becf-4e6e-8164-54ef8c80c7f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:46.319601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b09fec-becf-4e6e-8164-54ef8c80c7f3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:46.325111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 431/883] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Jul 14 10:01:46 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:46.430382 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cc1514-995d-4cc6-b9ef-499a2c92a032 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:46.469389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220741c7-567e-46a8-a5cd-220beb371f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:46.470293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220741c7-567e-46a8-a5cd-220beb371f33 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:46.470579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220741c7-567e-46a8-a5cd-220beb371f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:46.470854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220741c7-567e-46a8-a5cd-220beb371f33 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:46.471526 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220741c7-567e-46a8-a5cd-220beb371f33 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:46.472908 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220741c7-567e-46a8-a5cd-220beb371f33 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-220741c7-567e-46a8-a5cd-220beb371f33 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:46.486384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:46.487444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:46.488293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:46.488293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:46.488293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:46.488293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:46.488293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:46.488293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:46.488293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:46.488293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:46.488293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:46.488293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:46.488293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:46.488293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:46.488293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:01:46.488293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:46.488293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 453/884] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:01:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:46.495736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b09fec-becf-4e6e-8164-54ef8c80c7f3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:46.555633 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:46.611125 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b09fec-becf-4e6e-8164-54ef8c80c7f3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:46.648979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0cda5-1762-41c8-948f-eb715682dd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:46.649659 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0cda5-1762-41c8-948f-eb715682dd40 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:46.649939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0cda5-1762-41c8-948f-eb715682dd40 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:46.650280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0cda5-1762-41c8-948f-eb715682dd40 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:46.650996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0cda5-1762-41c8-948f-eb715682dd40 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:46.651356 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0cda5-1762-41c8-948f-eb715682dd40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:46.652779 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde0cda5-1762-41c8-948f-eb715682dd40 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:46.658207 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 432/885] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:01:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:46.665270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220741c7-567e-46a8-a5cd-220beb371f33 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:46.715011 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:46.781203 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220741c7-567e-46a8-a5cd-220beb371f33 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:46.816343 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f24fd-a776-447e-a1a9-3c99b545c08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:46.817320 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f24fd-a776-447e-a1a9-3c99b545c08f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:46.817746 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f24fd-a776-447e-a1a9-3c99b545c08f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:46.818160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585f24fd-a776-447e-a1a9-3c99b545c08f None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:46.819058 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f24fd-a776-447e-a1a9-3c99b545c08f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:46.819526 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f24fd-a776-447e-a1a9-3c99b545c08f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:46.821049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585f24fd-a776-447e-a1a9-3c99b545c08f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:46.826624 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 454/886] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:01:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:46.833025 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0cda5-1762-41c8-948f-eb715682dd40 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:46.940625 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde0cda5-1762-41c8-948f-eb715682dd40 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:46.955252 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:46.976175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf494e7b-a1d3-42df-8464-1e027e9f4406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:46.976831 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf494e7b-a1d3-42df-8464-1e027e9f4406 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:46.977143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf494e7b-a1d3-42df-8464-1e027e9f4406 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:46.977417 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf494e7b-a1d3-42df-8464-1e027e9f4406 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:46.978321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf494e7b-a1d3-42df-8464-1e027e9f4406 None admin] RBAC: Adding query filter params (interface=public, service_id=39e29318c3764ba4a18e3091ead16e72, region_id=RegionOne) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:46.978630 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf494e7b-a1d3-42df-8464-1e027e9f4406 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:46.979988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf494e7b-a1d3-42df-8464-1e027e9f4406 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:46.985792 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 433/887] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Jul 14 10:01:46 2022] GET /identity/v3/endpoints?service_id=39e29318c3764ba4a18e3091ead16e72&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:46.991350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f24fd-a776-447e-a1a9-3c99b545c08f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:47.099628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585f24fd-a776-447e-a1a9-3c99b545c08f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:47.102341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:47.137119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e714145-630a-455c-b510-790b34ff67a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:47.138137 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e714145-630a-455c-b510-790b34ff67a9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:47.138612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e714145-630a-455c-b510-790b34ff67a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:47.139075 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e714145-630a-455c-b510-790b34ff67a9 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:47.139986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e714145-630a-455c-b510-790b34ff67a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:47.141590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e714145-630a-455c-b510-790b34ff67a9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:47.147773 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 455/888] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Jul 14 10:01:46 2022] GET /identity/v3/services => generated 1268 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:01:47.339180 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de630ad9-ba6b-4d86-9cbc-102943d2bc3a None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:47.339717 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de630ad9-ba6b-4d86-9cbc-102943d2bc3a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:47.339799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de630ad9-ba6b-4d86-9cbc-102943d2bc3a None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:47.341412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 434/889] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:47.346510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c5127-ff54-41b4-b23e-3737b295a31f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:47.346899 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c5127-ff54-41b4-b23e-3737b295a31f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:47.347083 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c5127-ff54-41b4-b23e-3737b295a31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:47.364778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-9e0c5127-ff54-41b4-b23e-3737b295a31f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:47.365985 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9e0c5127-ff54-41b4-b23e-3737b295a31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:47.434944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 456/890] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:47.457522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de630ad9-ba6b-4d86-9cbc-102943d2bc3a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:47.577005 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de630ad9-ba6b-4d86-9cbc-102943d2bc3a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:47.613063 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8527a-7430-45a3-970b-1330f3292522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:47.613918 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8527a-7430-45a3-970b-1330f3292522 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:47.614236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8527a-7430-45a3-970b-1330f3292522 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:47.614534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8527a-7430-45a3-970b-1330f3292522 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:47.615156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8527a-7430-45a3-970b-1330f3292522 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:47.616515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8527a-7430-45a3-970b-1330f3292522 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-0fe8527a-7430-45a3-970b-1330f3292522 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:47.629936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:47.630939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:47.631811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 435/891] 10.222.0.61 () {56 vars in 1132 bytes} [Thu Jul 14 10:01:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:47.637380 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c5127-ff54-41b4-b23e-3737b295a31f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:47.734019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:47.741618 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c5127-ff54-41b4-b23e-3737b295a31f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:47.775804 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49f9fd-c98a-4cad-8a50-13d36c007c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:47.776463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49f9fd-c98a-4cad-8a50-13d36c007c18 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:47.776739 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49f9fd-c98a-4cad-8a50-13d36c007c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:47.777029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea49f9fd-c98a-4cad-8a50-13d36c007c18 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:47.777733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49f9fd-c98a-4cad-8a50-13d36c007c18 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:47.778035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49f9fd-c98a-4cad-8a50-13d36c007c18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:47.779374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea49f9fd-c98a-4cad-8a50-13d36c007c18 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:47.784816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 457/892] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Jul 14 10:01:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:47.790443 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8527a-7430-45a3-970b-1330f3292522 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:47.837192 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:47.898226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8527a-7430-45a3-970b-1330f3292522 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:47.933801 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0867d3-e194-47d2-ac5d-a97981a77907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:47.934472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0867d3-e194-47d2-ac5d-a97981a77907 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:47.934751 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0867d3-e194-47d2-ac5d-a97981a77907 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:47.935023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0867d3-e194-47d2-ac5d-a97981a77907 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:47.935731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0867d3-e194-47d2-ac5d-a97981a77907 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:47.936038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0867d3-e194-47d2-ac5d-a97981a77907 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:47.937434 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0867d3-e194-47d2-ac5d-a97981a77907 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:47.943078 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 436/893] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Jul 14 10:01:47 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:47.948568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49f9fd-c98a-4cad-8a50-13d36c007c18 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:47.969138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649d150-1a6e-4a0b-80cb-c88e9fa72efe None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:47.970006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649d150-1a6e-4a0b-80cb-c88e9fa72efe None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:47.970244 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649d150-1a6e-4a0b-80cb-c88e9fa72efe None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:47.971837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 437/894] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:01:47.976666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b1188-1896-42bb-b6aa-24501057aa0c None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:47.976966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b1188-1896-42bb-b6aa-24501057aa0c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:47.977261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4b1188-1896-42bb-b6aa-24501057aa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:47.995893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-fc4b1188-1896-42bb-b6aa-24501057aa0c None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:47.997435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fc4b1188-1896-42bb-b6aa-24501057aa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:48.071388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea49f9fd-c98a-4cad-8a50-13d36c007c18 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:48.084306 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 438/895] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:47 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:01:48.108692 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c74a2e-547e-4834-bc2b-7802b4526390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:48.109287 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b1188-1896-42bb-b6aa-24501057aa0c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:48.109409 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c74a2e-547e-4834-bc2b-7802b4526390 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:48.109579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c74a2e-547e-4834-bc2b-7802b4526390 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:48.109874 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c74a2e-547e-4834-bc2b-7802b4526390 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:48.110676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c74a2e-547e-4834-bc2b-7802b4526390 None admin] RBAC: Adding query filter params (interface=public, service_id=912f6cec859c45a5835af515e29949ed, region_id=RegionOne) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:48.110989 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c74a2e-547e-4834-bc2b-7802b4526390 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:48.112678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c74a2e-547e-4834-bc2b-7802b4526390 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:48.119757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 458/896] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Jul 14 10:01:47 2022] GET /identity/v3/endpoints?service_id=912f6cec859c45a5835af515e29949ed&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:48.126008 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c74a2e-547e-4834-bc2b-7802b4526390 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:48.151995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:48.161260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:48.221580 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4b1188-1896-42bb-b6aa-24501057aa0c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:48.262531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c74a2e-547e-4834-bc2b-7802b4526390 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:48.264789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee70ed-5934-41c9-93a6-c241ac18658e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:48.265906 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee70ed-5934-41c9-93a6-c241ac18658e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:48.266449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee70ed-5934-41c9-93a6-c241ac18658e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:48.266913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee70ed-5934-41c9-93a6-c241ac18658e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:48.284960 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee70ed-5934-41c9-93a6-c241ac18658e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-54ee70ed-5934-41c9-93a6-c241ac18658e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:48.293282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:48.294251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:48.295105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:48.295105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:48.295105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:48.295105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:48.295105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:48.295105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:48.295105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:48.295105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:48.295105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:48.295105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:48.295105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:01:48.295105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:48.299973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 439/897] 10.222.0.61 () {56 vars in 1120 bytes} [Thu Jul 14 10:01:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:48.300992 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee70ed-5934-41c9-93a6-c241ac18658e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:48.318611 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc7594-4407-4d95-9663-8af96ed7c28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:48.320179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc7594-4407-4d95-9663-8af96ed7c28f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:48.320811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc7594-4407-4d95-9663-8af96ed7c28f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:48.321303 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc7594-4407-4d95-9663-8af96ed7c28f None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:48.322442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc7594-4407-4d95-9663-8af96ed7c28f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:48.324408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cc7594-4407-4d95-9663-8af96ed7c28f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:48.333919 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 459/898] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Jul 14 10:01:48 2022] GET /identity/v3/services => generated 1268 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:01:48.412571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:48.420569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee70ed-5934-41c9-93a6-c241ac18658e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:48.460024 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a77620-51e9-4c90-b1d1-ceb3ab3176c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:48.461051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a77620-51e9-4c90-b1d1-ceb3ab3176c5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:48.461510 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a77620-51e9-4c90-b1d1-ceb3ab3176c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:48.461955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a77620-51e9-4c90-b1d1-ceb3ab3176c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:48.462920 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a77620-51e9-4c90-b1d1-ceb3ab3176c5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:48.463405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a77620-51e9-4c90-b1d1-ceb3ab3176c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:48.465006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a77620-51e9-4c90-b1d1-ceb3ab3176c5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:48.475653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 440/899] 10.222.0.61 () {56 vars in 1115 bytes} [Thu Jul 14 10:01:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:48.482069 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc7594-4407-4d95-9663-8af96ed7c28f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:48.509644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373608d1-6a9d-44e5-84ce-2cbee144913a None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:48.510155 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373608d1-6a9d-44e5-84ce-2cbee144913a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:48.510832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373608d1-6a9d-44e5-84ce-2cbee144913a None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:48.513348 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 441/900] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:01:48.519617 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80106474-4661-49d2-b2c4-253a127440aa None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:48.519897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80106474-4661-49d2-b2c4-253a127440aa None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:48.520169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80106474-4661-49d2-b2c4-253a127440aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:48.537951 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-80106474-4661-49d2-b2c4-253a127440aa None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:48.539137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-80106474-4661-49d2-b2c4-253a127440aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:48.591876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:48.603166 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc7594-4407-4d95-9663-8af96ed7c28f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:48.612756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 442/901] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:48 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:48.635980 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80106474-4661-49d2-b2c4-253a127440aa None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:48.645284 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44daaef-aa7d-4c74-93bb-293c857c13ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:48.646452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44daaef-aa7d-4c74-93bb-293c857c13ca None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:48.646988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44daaef-aa7d-4c74-93bb-293c857c13ca None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:48.647571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44daaef-aa7d-4c74-93bb-293c857c13ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:48.659999 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44daaef-aa7d-4c74-93bb-293c857c13ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:48.661848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44daaef-aa7d-4c74-93bb-293c857c13ca None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:48.680184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-a44daaef-aa7d-4c74-93bb-293c857c13ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:48.680184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:48.680184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:48.680184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:48.680184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:48.680184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:48.680184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:48.680184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:48.680184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:48.680184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:48.680184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:48.680184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:48.680184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:48.680184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:48.680184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:48.681151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:48.682066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:48.682066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:48.682066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:48.682066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:48.684366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 460/902] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:48.690654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44daaef-aa7d-4c74-93bb-293c857c13ca None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:48.751284 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:48.763882 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80106474-4661-49d2-b2c4-253a127440aa None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:48.808724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44daaef-aa7d-4c74-93bb-293c857c13ca None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:48.814497 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed71ab-ff80-463c-a739-e8e68d4f3ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:48.815481 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed71ab-ff80-463c-a739-e8e68d4f3ec3 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:48.815962 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed71ab-ff80-463c-a739-e8e68d4f3ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:48.816433 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed71ab-ff80-463c-a739-e8e68d4f3ec3 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:48.817956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed71ab-ff80-463c-a739-e8e68d4f3ec3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:48.820101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed71ab-ff80-463c-a739-e8e68d4f3ec3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:48.845385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 443/903] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Jul 14 10:01:48 2022] POST /identity/v3/services => generated 241 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:01:48.858054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72e8b1-6041-4606-863b-c735a3096395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:48.858827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72e8b1-6041-4606-863b-c735a3096395 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:48.859108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72e8b1-6041-4606-863b-c735a3096395 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:48.859376 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a72e8b1-6041-4606-863b-c735a3096395 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:48.860160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72e8b1-6041-4606-863b-c735a3096395 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:48.860460 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72e8b1-6041-4606-863b-c735a3096395 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:48.862051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a72e8b1-6041-4606-863b-c735a3096395 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:48.872388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:48.876036 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 461/904] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:48.881616 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed71ab-ff80-463c-a739-e8e68d4f3ec3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:49.001734 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed71ab-ff80-463c-a739-e8e68d4f3ec3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:49.042255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc738d-3f5c-4bce-a7d7-d5110e230961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:49.043408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc738d-3f5c-4bce-a7d7-d5110e230961 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:49.043687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc738d-3f5c-4bce-a7d7-d5110e230961 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:49.043958 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc738d-3f5c-4bce-a7d7-d5110e230961 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:49.053630 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc738d-3f5c-4bce-a7d7-d5110e230961 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:49.055064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc738d-3f5c-4bce-a7d7-d5110e230961 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-f7fc738d-3f5c-4bce-a7d7-d5110e230961 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:49.069996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:49.070938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:49.071873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:49.071873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:49.071873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:49.071873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:49.071873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:49.071873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:49.071873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:49.071873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:49.071873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:01:49.071873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:49.071873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 444/905] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Jul 14 10:01:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:49.076854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72e8b1-6041-4606-863b-c735a3096395 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:49.181530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a72e8b1-6041-4606-863b-c735a3096395 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:49.220140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363c93e-a504-47c8-85ae-6eafd275fb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:49.220814 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363c93e-a504-47c8-85ae-6eafd275fb2a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:49.221123 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363c93e-a504-47c8-85ae-6eafd275fb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:49.221396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363c93e-a504-47c8-85ae-6eafd275fb2a None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:49.222242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363c93e-a504-47c8-85ae-6eafd275fb2a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:49.222551 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363c93e-a504-47c8-85ae-6eafd275fb2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:49.223983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363c93e-a504-47c8-85ae-6eafd275fb2a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:49.242704 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 462/906] 10.222.0.61 () {56 vars in 1149 bytes} [Thu Jul 14 10:01:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:49.255532 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc738d-3f5c-4bce-a7d7-d5110e230961 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:49.257870 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1b519-517e-46a0-a970-969c0a93f0b7 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:49.258159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1b519-517e-46a0-a970-969c0a93f0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:49.258435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1b519-517e-46a0-a970-969c0a93f0b7 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:49.260070 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 463/907] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:49.264631 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a6f09-c383-402e-b04d-0a5a0762628b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:49.264934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a6f09-c383-402e-b04d-0a5a0762628b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:49.265240 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48a6f09-c383-402e-b04d-0a5a0762628b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:49.270635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:49.285364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a48a6f09-c383-402e-b04d-0a5a0762628b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:49.286572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a48a6f09-c383-402e-b04d-0a5a0762628b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:49.362723 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:49.375440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc738d-3f5c-4bce-a7d7-d5110e230961 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:49.389006 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 464/908] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:01:49.415718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a6f09-c383-402e-b04d-0a5a0762628b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:49.421606 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff19f7-209b-4629-a80c-651cc87af795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:49.422975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff19f7-209b-4629-a80c-651cc87af795 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:49.423461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff19f7-209b-4629-a80c-651cc87af795 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:49.423962 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff19f7-209b-4629-a80c-651cc87af795 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:49.522976 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff19f7-209b-4629-a80c-651cc87af795 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:49.524335 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff19f7-209b-4629-a80c-651cc87af795 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:49.527907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48a6f09-c383-402e-b04d-0a5a0762628b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:49.539200 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:49.568740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c19c4-edcc-46ee-bf03-136d355fef91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:49.569598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c19c4-edcc-46ee-bf03-136d355fef91 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:49.569877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c19c4-edcc-46ee-bf03-136d355fef91 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:49.570148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c19c4-edcc-46ee-bf03-136d355fef91 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:49.570853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c19c4-edcc-46ee-bf03-136d355fef91 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:49.572192 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361c19c4-edcc-46ee-bf03-136d355fef91 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-361c19c4-edcc-46ee-bf03-136d355fef91 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:49.584807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:49.586042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:49.587002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:49.587002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:49.587002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:49.587002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:49.587002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:49.587002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:49.587002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:49.587002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:49.587002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:49.587002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:49.587002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:49.587002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:49.587002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 14 10:01:49.587002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:49.591003 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 465/909] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:01:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:49.591914 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c19c4-edcc-46ee-bf03-136d355fef91 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:49.645560 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:49.673997 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 445/910] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:49 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:01:49.674919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff19f7-209b-4629-a80c-651cc87af795 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:49.706880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c19c4-edcc-46ee-bf03-136d355fef91 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:49.740400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccab2b-5db0-4d7a-89c9-6ae059f0c9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:49.741223 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccab2b-5db0-4d7a-89c9-6ae059f0c9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:49.741501 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccab2b-5db0-4d7a-89c9-6ae059f0c9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:49.741776 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ccab2b-5db0-4d7a-89c9-6ae059f0c9b7 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:49.742574 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ccab2b-5db0-4d7a-89c9-6ae059f0c9b7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:49.742836 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ccab2b-5db0-4d7a-89c9-6ae059f0c9b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:49.744201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ccab2b-5db0-4d7a-89c9-6ae059f0c9b7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:49.753294 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 466/911] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:01:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:49.755657 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccab2b-5db0-4d7a-89c9-6ae059f0c9b7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:49.787555 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff19f7-209b-4629-a80c-651cc87af795 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:49.797662 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:49.826793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a0ac6-4580-4f8a-802d-e183147bd646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:49.827866 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a0ac6-4580-4f8a-802d-e183147bd646 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:49.828961 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a0ac6-4580-4f8a-802d-e183147bd646 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:49.829413 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a0ac6-4580-4f8a-802d-e183147bd646 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:49.840651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a0ac6-4580-4f8a-802d-e183147bd646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:49.842435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a0ac6-4580-4f8a-802d-e183147bd646 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:49.857890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ccab2b-5db0-4d7a-89c9-6ae059f0c9b7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:49.862968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-604a0ac6-4580-4f8a-802d-e183147bd646 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:49.862968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:49.862968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:49.862968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:49.862968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:49.862968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:49.862968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:49.862968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:49.862968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:49.862968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:49.862968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:49.862968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:49.862968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:49.862968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:49.862968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:49.864139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:49.865288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:49.865288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:49.865288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:49.865288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:49.867320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 446/912] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:49.873225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a0ac6-4580-4f8a-802d-e183147bd646 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:49.876554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:49.897938 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b20f5-d825-4ac8-b5d5-58e327f85217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:49.898609 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b20f5-d825-4ac8-b5d5-58e327f85217 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:49.898884 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b20f5-d825-4ac8-b5d5-58e327f85217 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:49.899153 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b20f5-d825-4ac8-b5d5-58e327f85217 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:49.899865 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b20f5-d825-4ac8-b5d5-58e327f85217 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:49.900167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b20f5-d825-4ac8-b5d5-58e327f85217 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:49.901558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5b20f5-d825-4ac8-b5d5-58e327f85217 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:49.911525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 467/913] 10.222.0.61 () {56 vars in 1114 bytes} [Thu Jul 14 10:01:49 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:49.912365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b20f5-d825-4ac8-b5d5-58e327f85217 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:49.991076 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a0ac6-4580-4f8a-802d-e183147bd646 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:50.015691 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b20f5-d825-4ac8-b5d5-58e327f85217 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:50.033220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb5329-4dc9-4b84-9728-5ce27d3312b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:50.034367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb5329-4dc9-4b84-9728-5ce27d3312b8 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:50.034931 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb5329-4dc9-4b84-9728-5ce27d3312b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:50.035369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcb5329-4dc9-4b84-9728-5ce27d3312b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:50.036276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb5329-4dc9-4b84-9728-5ce27d3312b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:50.036738 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb5329-4dc9-4b84-9728-5ce27d3312b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:50.038573 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcb5329-4dc9-4b84-9728-5ce27d3312b8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:50.053061 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 447/914] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:50.057369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445a9e8-309f-46e9-acf4-c975eb50dc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:50.059014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb5329-4dc9-4b84-9728-5ce27d3312b8 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:50.059651 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445a9e8-309f-46e9-acf4-c975eb50dc3e None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:50.060139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445a9e8-309f-46e9-acf4-c975eb50dc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:50.060979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445a9e8-309f-46e9-acf4-c975eb50dc3e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:50.061934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445a9e8-309f-46e9-acf4-c975eb50dc3e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:50.065196 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445a9e8-309f-46e9-acf4-c975eb50dc3e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:50.088275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:50.100530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 468/915] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Jul 14 10:01:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:01:50.186651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcb5329-4dc9-4b84-9728-5ce27d3312b8 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:50.231868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41740e30-916d-4dce-96ba-bb0ddff9c273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:50.232723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41740e30-916d-4dce-96ba-bb0ddff9c273 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:50.233094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41740e30-916d-4dce-96ba-bb0ddff9c273 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:50.233365 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41740e30-916d-4dce-96ba-bb0ddff9c273 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:50.253733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41740e30-916d-4dce-96ba-bb0ddff9c273 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:50.255218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41740e30-916d-4dce-96ba-bb0ddff9c273 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:50.270372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-41740e30-916d-4dce-96ba-bb0ddff9c273 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:50.270372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:50.270372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:50.270372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:50.270372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:50.270372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:50.270372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:50.270372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:50.270372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:50.270372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:50.270372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:50.270372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:50.270372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:50.270372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:50.270372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:50.271312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:50.272237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:50.272237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:50.272237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:50.272237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:50.272237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:50.272237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:50.272237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:50.272237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:50.272237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 448/916] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:50.277654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445a9e8-309f-46e9-acf4-c975eb50dc3e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:50.315688 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:50.390697 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445a9e8-309f-46e9-acf4-c975eb50dc3e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:50.426034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f660be6-7cd4-4050-ab1e-7fccc07eb61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:50.426938 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f660be6-7cd4-4050-ab1e-7fccc07eb61d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:50.427222 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f660be6-7cd4-4050-ab1e-7fccc07eb61d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:50.427489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f660be6-7cd4-4050-ab1e-7fccc07eb61d None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:50.428279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f660be6-7cd4-4050-ab1e-7fccc07eb61d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:50.428581 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f660be6-7cd4-4050-ab1e-7fccc07eb61d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:50.430222 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f660be6-7cd4-4050-ab1e-7fccc07eb61d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:50.443484 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 469/917] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:50.449350 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41740e30-916d-4dce-96ba-bb0ddff9c273 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:50.493673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75839d7e-ba27-4e9f-b861-717aa970f412 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:50.493983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75839d7e-ba27-4e9f-b861-717aa970f412 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:50.494205 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75839d7e-ba27-4e9f-b861-717aa970f412 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:50.495896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 470/918] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:01:50.502234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361964c-5186-4742-b6d4-dedbd5174a5f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:50.502514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361964c-5186-4742-b6d4-dedbd5174a5f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:50.502788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361964c-5186-4742-b6d4-dedbd5174a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:50.523079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d361964c-5186-4742-b6d4-dedbd5174a5f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:50.524849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d361964c-5186-4742-b6d4-dedbd5174a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:50.538814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:50.572627 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41740e30-916d-4dce-96ba-bb0ddff9c273 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:50.612697 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 471/919] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:50.622744 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb74fb-9e35-44e9-a779-c555bc5b1ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:50.623814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb74fb-9e35-44e9-a779-c555bc5b1ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:50.624354 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb74fb-9e35-44e9-a779-c555bc5b1ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:50.624852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadb74fb-9e35-44e9-a779-c555bc5b1ed4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:50.626184 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb74fb-9e35-44e9-a779-c555bc5b1ed4 None admin] RBAC: Adding query filter params (role.id=9cb5d3eed6dc41589bac547cebf73cdb, scope.project.id=57662256880f43018039715a755c087c, user.id=dd2b9ad7efa848938627385b411727ee) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:50.626723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb74fb-9e35-44e9-a779-c555bc5b1ed4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:50.628383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadb74fb-9e35-44e9-a779-c555bc5b1ed4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:50.635137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 449/920] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:50 2022] GET /identity/v3/role_assignments?user.id=dd2b9ad7efa848938627385b411727ee&scope.project.id=57662256880f43018039715a755c087c&role.id=9cb5d3eed6dc41589bac547cebf73cdb => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:50.642252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361964c-5186-4742-b6d4-dedbd5174a5f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:50.671023 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:50.753365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361964c-5186-4742-b6d4-dedbd5174a5f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:50.795480 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9d244-1552-4d83-a78f-a718e9342746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:50.796309 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9d244-1552-4d83-a78f-a718e9342746 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:50.796591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9d244-1552-4d83-a78f-a718e9342746 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:50.797443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9d244-1552-4d83-a78f-a718e9342746 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:50.798132 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9d244-1552-4d83-a78f-a718e9342746 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:50.799524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9d244-1552-4d83-a78f-a718e9342746 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-86f9d244-1552-4d83-a78f-a718e9342746 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:50.812784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:50.813815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:50.814678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 472/921] 10.222.0.61 () {56 vars in 1132 bytes} [Thu Jul 14 10:01:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:50.820176 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb74fb-9e35-44e9-a779-c555bc5b1ed4 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:50.931509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadb74fb-9e35-44e9-a779-c555bc5b1ed4 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:50.967187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07224dc9-2e1b-4697-becd-0eae14766aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:50.967852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07224dc9-2e1b-4697-becd-0eae14766aae None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:50.968129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07224dc9-2e1b-4697-becd-0eae14766aae None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:50.968401 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07224dc9-2e1b-4697-becd-0eae14766aae None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:50.969707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07224dc9-2e1b-4697-becd-0eae14766aae None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:50.970008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07224dc9-2e1b-4697-becd-0eae14766aae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:50.971367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07224dc9-2e1b-4697-becd-0eae14766aae None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:50.976737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 450/922] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Jul 14 10:01:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:50.982380 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9d244-1552-4d83-a78f-a718e9342746 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:50.990013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab43045-4eb6-4cbc-950a-433eef98807e None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:50.990848 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab43045-4eb6-4cbc-950a-433eef98807e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:50.991333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab43045-4eb6-4cbc-950a-433eef98807e None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:50.993500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 451/923] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:51.002538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c4939-a24a-4766-aed7-9e2c89b619b0 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:51.002828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c4939-a24a-4766-aed7-9e2c89b619b0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:51.003106 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c4939-a24a-4766-aed7-9e2c89b619b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:51.009365 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:51.021870 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-cc5c4939-a24a-4766-aed7-9e2c89b619b0 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:51.023066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cc5c4939-a24a-4766-aed7-9e2c89b619b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:51.105295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:51.106330 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 452/924] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:01:51.114098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9d244-1552-4d83-a78f-a718e9342746 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:51.127111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c4939-a24a-4766-aed7-9e2c89b619b0 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:51.160352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b53d3-15c4-4eb7-9cad-4d95f47b032e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:51.161519 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b53d3-15c4-4eb7-9cad-4d95f47b032e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:51.162034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b53d3-15c4-4eb7-9cad-4d95f47b032e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:51.162510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b53d3-15c4-4eb7-9cad-4d95f47b032e None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:51.163425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b53d3-15c4-4eb7-9cad-4d95f47b032e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:51.163982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b53d3-15c4-4eb7-9cad-4d95f47b032e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:51.165684 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b53d3-15c4-4eb7-9cad-4d95f47b032e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:51.175979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 473/925] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Jul 14 10:01:50 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:51.177524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b53d3-15c4-4eb7-9cad-4d95f47b032e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:51.232402 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c4939-a24a-4766-aed7-9e2c89b619b0 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:51.268685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936715e8-d7c2-4e2a-b255-52216608885a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:51.269528 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936715e8-d7c2-4e2a-b255-52216608885a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:51.270020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936715e8-d7c2-4e2a-b255-52216608885a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:51.270129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936715e8-d7c2-4e2a-b255-52216608885a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:51.270701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936715e8-d7c2-4e2a-b255-52216608885a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:51.272071 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936715e8-d7c2-4e2a-b255-52216608885a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:51.273965 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-936715e8-d7c2-4e2a-b255-52216608885a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:51.285525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:51.286794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:51.287846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:51.287846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:51.287846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:51.287846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:51.287846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:51.287846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:51.287846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:51.287846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:51.287846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:51.287846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:51.287846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:51.287846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:51.287846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:01:51.287846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:51.291324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 453/926] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Jul 14 10:01:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:51.297254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936715e8-d7c2-4e2a-b255-52216608885a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:51.306742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b53d3-15c4-4eb7-9cad-4d95f47b032e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:51.353644 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700d16c-c602-4920-b166-cf744c6c966d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:51.354674 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700d16c-c602-4920-b166-cf744c6c966d None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:51.355142 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700d16c-c602-4920-b166-cf744c6c966d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:51.356163 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700d16c-c602-4920-b166-cf744c6c966d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:51.357306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700d16c-c602-4920-b166-cf744c6c966d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:51.359485 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700d16c-c602-4920-b166-cf744c6c966d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:51.393464 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 474/927] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Jul 14 10:01:51 2022] POST /identity/v3/endpoints => generated 347 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:01:51.420568 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936715e8-d7c2-4e2a-b255-52216608885a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:51.467003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bd3b9-9138-4f25-a4a4-27ee5f63d9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:51.467782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bd3b9-9138-4f25-a4a4-27ee5f63d9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:51.467975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bd3b9-9138-4f25-a4a4-27ee5f63d9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:51.468215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67bd3b9-9138-4f25-a4a4-27ee5f63d9f3 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:51.468977 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67bd3b9-9138-4f25-a4a4-27ee5f63d9f3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:51.469288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67bd3b9-9138-4f25-a4a4-27ee5f63d9f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:51.470827 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67bd3b9-9138-4f25-a4a4-27ee5f63d9f3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:51.477321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 454/928] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Jul 14 10:01:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:51.490236 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700d16c-c602-4920-b166-cf744c6c966d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:51.504174 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:51.607551 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700d16c-c602-4920-b166-cf744c6c966d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:51.643776 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76cb930-fda4-40cc-81fb-ab4606950690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:51.645145 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cb930-fda4-40cc-81fb-ab4606950690 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:51.645609 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cb930-fda4-40cc-81fb-ab4606950690 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:51.646029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76cb930-fda4-40cc-81fb-ab4606950690 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:51.646890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76cb930-fda4-40cc-81fb-ab4606950690 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:51.647344 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76cb930-fda4-40cc-81fb-ab4606950690 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:51.648863 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76cb930-fda4-40cc-81fb-ab4606950690 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:51.654625 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 475/929] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Jul 14 10:01:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:51.660612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bd3b9-9138-4f25-a4a4-27ee5f63d9f3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:51.723759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:51.771052 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67bd3b9-9138-4f25-a4a4-27ee5f63d9f3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:51.809824 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7510dd0-5ba5-47a0-9b55-e6354f564296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:51.810774 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7510dd0-5ba5-47a0-9b55-e6354f564296 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:51.811207 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7510dd0-5ba5-47a0-9b55-e6354f564296 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:51.811628 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7510dd0-5ba5-47a0-9b55-e6354f564296 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:51.813238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7510dd0-5ba5-47a0-9b55-e6354f564296 None admin] RBAC: Adding query filter params (interface=public, service_id=01bba6c0a39e4b0e949a5f01b975ce5c, region_id=RegionOne) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:51.813704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7510dd0-5ba5-47a0-9b55-e6354f564296 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:51.815240 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7510dd0-5ba5-47a0-9b55-e6354f564296 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:51.821370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 455/930] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Jul 14 10:01:51 2022] GET /identity/v3/endpoints?service_id=01bba6c0a39e4b0e949a5f01b975ce5c&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:51.827024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76cb930-fda4-40cc-81fb-ab4606950690 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:51.899934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:51.932555 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76cb930-fda4-40cc-81fb-ab4606950690 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:51.968136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c8acfa-76ac-4482-b619-ade2eb6391d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:51.969154 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8acfa-76ac-4482-b619-ade2eb6391d0 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:51.969585 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8acfa-76ac-4482-b619-ade2eb6391d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:51.970030 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c8acfa-76ac-4482-b619-ade2eb6391d0 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:51.970877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8acfa-76ac-4482-b619-ade2eb6391d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:51.972246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c8acfa-76ac-4482-b619-ade2eb6391d0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:51.980251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 476/931] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Jul 14 10:01:51 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:01:52.320083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f17112-f68b-461a-95ab-c648f8bf21cf None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:52.320407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f17112-f68b-461a-95ab-c648f8bf21cf None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:52.320698 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f17112-f68b-461a-95ab-c648f8bf21cf None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:52.322366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 456/932] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:52.327462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a3b3f-33c0-4f0c-9ea6-015c1f840129 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:52.327809 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a3b3f-33c0-4f0c-9ea6-015c1f840129 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:52.328610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47a3b3f-33c0-4f0c-9ea6-015c1f840129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:52.345800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e47a3b3f-33c0-4f0c-9ea6-015c1f840129 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:52.346991 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e47a3b3f-33c0-4f0c-9ea6-015c1f840129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:52.418469 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 477/933] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:52.442892 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f17112-f68b-461a-95ab-c648f8bf21cf None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:52.554666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f17112-f68b-461a-95ab-c648f8bf21cf None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:52.580393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:52.594434 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3221d-74f6-4923-b387-be2d0a758bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:52.595419 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3221d-74f6-4923-b387-be2d0a758bd1 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:52.595903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3221d-74f6-4923-b387-be2d0a758bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:52.596386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3221d-74f6-4923-b387-be2d0a758bd1 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:52.597340 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3221d-74f6-4923-b387-be2d0a758bd1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:52.599003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3221d-74f6-4923-b387-be2d0a758bd1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:52.600613 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: WARNING keystone.server.flask.common [None req-1da3221d-74f6-4923-b387-be2d0a758bd1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 14 10:01:52.619552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.sql.core [None req-1da3221d-74f6-4923-b387-be2d0a758bd1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 10:01:52.619552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 10:01:52.619552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [parameters: {'id': 'f0960abf3cf649a4b825f6299c510fd0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 10:01:52.619552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80320) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 10:01:52.638416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-1da3221d-74f6-4923-b387-be2d0a758bd1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f0960abf3cf649a4b825f6299c510fd0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f0960abf3cf649a4b825f6299c510fd0. Jul 14 10:01:52.638416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:52.638416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:01:52.638416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:01:52.638416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:01:52.638416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:01:52.638416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:01:52.638416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:01:52.638416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:01:52.638416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:01:52.638416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:01:52.638416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:01:52.638416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result.read() Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 14 10:01:52.639307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application yield resource Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise self.value Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application session.commit() Jul 14 10:01:52.640346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.session.flush() Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application compat.raise_( Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 10:01:52.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 10:01:52.643167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 10:01:52.643167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 10:01:52.643167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 10:01:52.643167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 10:01:52.643167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 14 10:01:52.643167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 14 10:01:52.643167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 10:01:52.643167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 10:01:52.643167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 10:01:52.643167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 10:01:52.643167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 10:01:52.643167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 10:01:52.643167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 10:01:52.643167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 10:01:52.643167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result.read() Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:01:52.645137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:01:52.647291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:01:52.647291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:01:52.647291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:01:52.647291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:01:52.647291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:01:52.647291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:01:52.647291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 14 10:01:52.647291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 14 10:01:52.647291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [parameters: {'id': 'f0960abf3cf649a4b825f6299c510fd0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 14 10:01:52.647291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 10:01:52.647291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:52.647291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:52.647291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:52.647291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:52.647291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:52.649202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:52.651316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:52.651316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 14 10:01:52.651316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 14 10:01:52.651316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:52.651316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:52.651316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 14 10:01:52.651316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 14 10:01:52.651316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f0960abf3cf649a4b825f6299c510fd0. Jul 14 10:01:52.651316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:52.654102 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 457/934] 10.222.0.61 () {60 vars in 1140 bytes} [Thu Jul 14 10:01:52 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:52.661077 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a3b3f-33c0-4f0c-9ea6-015c1f840129 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:52.707979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b3e1b-1041-4092-b959-d0748eab08a5 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:52.708659 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b3e1b-1041-4092-b959-d0748eab08a5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:52.709203 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b3e1b-1041-4092-b959-d0748eab08a5 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:52.714037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 458/935] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:01:52.720954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb86cf2-0604-40ac-8e62-5840a33dc6f9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:52.720954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb86cf2-0604-40ac-8e62-5840a33dc6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:52.720954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb86cf2-0604-40ac-8e62-5840a33dc6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:52.735230 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:52.741190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6cb86cf2-0604-40ac-8e62-5840a33dc6f9 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:52.742453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6cb86cf2-0604-40ac-8e62-5840a33dc6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:52.773956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47a3b3f-33c0-4f0c-9ea6-015c1f840129 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:52.809638 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacd192-454e-4f4a-aca9-06c03efccbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:52.810410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacd192-454e-4f4a-aca9-06c03efccbf2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:52.810702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacd192-454e-4f4a-aca9-06c03efccbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:52.810978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daacd192-454e-4f4a-aca9-06c03efccbf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:52.814065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 459/936] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:01:52.818379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacd192-454e-4f4a-aca9-06c03efccbf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:52.819818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daacd192-454e-4f4a-aca9-06c03efccbf2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:52.835807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-daacd192-454e-4f4a-aca9-06c03efccbf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:52.835807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:52.835807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:52.835807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:52.835807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:52.835807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:52.835807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:52.835807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:52.835807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:52.835807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:52.835807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:52.835807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:52.835807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:52.835807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:52.835807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:52.836838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:52.837850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:52.837850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:52.837850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:52.837850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:52.837850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:52.837850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:52.837850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:52.837850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:52.837850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 478/937] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:52.843023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb86cf2-0604-40ac-8e62-5840a33dc6f9 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:52.846097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacd192-454e-4f4a-aca9-06c03efccbf2 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:52.955885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daacd192-454e-4f4a-aca9-06c03efccbf2 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:52.975201 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb86cf2-0604-40ac-8e62-5840a33dc6f9 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:52.983536 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:52.989010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:52.996070 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9edca-791c-4f41-a928-85d484341900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:52.996961 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9edca-791c-4f41-a928-85d484341900 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:52.997315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9edca-791c-4f41-a928-85d484341900 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:52.997597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9edca-791c-4f41-a928-85d484341900 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:53.017245 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9edca-791c-4f41-a928-85d484341900 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:53.020772 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5449e1-7556-41b5-9616-fd47df95853b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:53.021630 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5449e1-7556-41b5-9616-fd47df95853b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:53.022078 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5449e1-7556-41b5-9616-fd47df95853b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:53.022505 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5449e1-7556-41b5-9616-fd47df95853b None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:53.023434 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5449e1-7556-41b5-9616-fd47df95853b None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:53.023897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5449e1-7556-41b5-9616-fd47df95853b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:53.025622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5449e1-7556-41b5-9616-fd47df95853b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-6ec9edca-791c-4f41-a928-85d484341900 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:53.026784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:53.027934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:53.028997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:53.028997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:53.028997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:53.028997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:53.028997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:53.028997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:53.028997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:53.028997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:53.028997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:53.028997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:53.028997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:01:53.028997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:53.036336 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 479/938] 10.222.0.61 () {56 vars in 1120 bytes} [Thu Jul 14 10:01:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:53.037302 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9edca-791c-4f41-a928-85d484341900 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:53.057051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 460/939] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Jul 14 10:01:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:53.152884 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9edca-791c-4f41-a928-85d484341900 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:53.193387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebec254-b9e9-4e01-b791-6dc2661f961d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:53.194064 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebec254-b9e9-4e01-b791-6dc2661f961d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:53.194341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebec254-b9e9-4e01-b791-6dc2661f961d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:53.194610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebec254-b9e9-4e01-b791-6dc2661f961d None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:53.195432 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebec254-b9e9-4e01-b791-6dc2661f961d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:53.195747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebec254-b9e9-4e01-b791-6dc2661f961d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:53.197156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebec254-b9e9-4e01-b791-6dc2661f961d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:53.206508 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 480/940] 10.222.0.61 () {56 vars in 1115 bytes} [Thu Jul 14 10:01:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:53.212366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5449e1-7556-41b5-9616-fd47df95853b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:53.324793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5449e1-7556-41b5-9616-fd47df95853b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:53.365020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ec5d8-4cf6-4378-85ea-455e2c9c7cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:53.365817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ec5d8-4cf6-4378-85ea-455e2c9c7cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:53.366096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ec5d8-4cf6-4378-85ea-455e2c9c7cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:53.366367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ec5d8-4cf6-4378-85ea-455e2c9c7cf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:53.377344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ec5d8-4cf6-4378-85ea-455e2c9c7cf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:53.378794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ec5d8-4cf6-4378-85ea-455e2c9c7cf5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:53.396292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-c74ec5d8-4cf6-4378-85ea-455e2c9c7cf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:53.396292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:53.396292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:53.396292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:53.396292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:53.396292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:53.396292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:53.396292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:53.396292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:53.396292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:53.396292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:53.396292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:53.396292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:53.396292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:53.396292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:53.397227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:53.398140 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:53.398140 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:53.398140 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:53.398140 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:53.398140 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 461/941] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:53.403507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebec254-b9e9-4e01-b791-6dc2661f961d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:53.454929 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:53.507084 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebec254-b9e9-4e01-b791-6dc2661f961d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:53.541700 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06643944-3339-4dcd-8793-cfcbbe5a6e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:53.542739 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06643944-3339-4dcd-8793-cfcbbe5a6e06 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:53.543231 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06643944-3339-4dcd-8793-cfcbbe5a6e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:53.543744 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06643944-3339-4dcd-8793-cfcbbe5a6e06 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:53.544672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06643944-3339-4dcd-8793-cfcbbe5a6e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:53.545208 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06643944-3339-4dcd-8793-cfcbbe5a6e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:53.547354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06643944-3339-4dcd-8793-cfcbbe5a6e06 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:53.560328 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 481/942] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:53.565683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ec5d8-4cf6-4378-85ea-455e2c9c7cf5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:53.591054 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:53.592200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87a775-1c89-4e3f-8d3e-81bfa17747a2 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:53.592707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87a775-1c89-4e3f-8d3e-81bfa17747a2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:53.593744 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87a775-1c89-4e3f-8d3e-81bfa17747a2 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:53.596348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 482/943] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:01:53.601690 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281505a-5a20-4f29-9e61-d454025f0fdd None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:53.602196 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281505a-5a20-4f29-9e61-d454025f0fdd None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:53.602684 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281505a-5a20-4f29-9e61-d454025f0fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:53.620887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f281505a-5a20-4f29-9e61-d454025f0fdd None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:53.622564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f281505a-5a20-4f29-9e61-d454025f0fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:53.675946 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ec5d8-4cf6-4378-85ea-455e2c9c7cf5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:53.700967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 483/944] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:53.711679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcedc8a-2170-4842-ade4-3615da554f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:53.712555 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcedc8a-2170-4842-ade4-3615da554f91 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:53.712846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcedc8a-2170-4842-ade4-3615da554f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:53.713153 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcedc8a-2170-4842-ade4-3615da554f91 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:53.721663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcedc8a-2170-4842-ade4-3615da554f91 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:53.721982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281505a-5a20-4f29-9e61-d454025f0fdd None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:53.723112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcedc8a-2170-4842-ade4-3615da554f91 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-adcedc8a-2170-4842-ade4-3615da554f91 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:53.748881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:53.750041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:53.751023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:53.751023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:53.751023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:53.751023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:53.751023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:53.751023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:53.751023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:53.751023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:53.751023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:01:53.751023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:53.753629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 462/945] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Jul 14 10:01:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:53.759970 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcedc8a-2170-4842-ade4-3615da554f91 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:53.772855 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:53.857425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281505a-5a20-4f29-9e61-d454025f0fdd None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:53.858502 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:53.880033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcedc8a-2170-4842-ade4-3615da554f91 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:53.906661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec9bcb-a863-4811-b6b7-4453e5f8c677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:53.908336 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9bcb-a863-4811-b6b7-4453e5f8c677 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:53.908765 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9bcb-a863-4811-b6b7-4453e5f8c677 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:53.909206 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec9bcb-a863-4811-b6b7-4453e5f8c677 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:53.910286 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec9bcb-a863-4811-b6b7-4453e5f8c677 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:53.912462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec9bcb-a863-4811-b6b7-4453e5f8c677 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:53.918780 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f282037-5eaf-4051-888e-38fadaf902bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:53.919626 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f282037-5eaf-4051-888e-38fadaf902bb None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:53.920016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f282037-5eaf-4051-888e-38fadaf902bb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:53.920398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f282037-5eaf-4051-888e-38fadaf902bb None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:53.921426 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f282037-5eaf-4051-888e-38fadaf902bb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:53.921845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f282037-5eaf-4051-888e-38fadaf902bb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:53.923393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f282037-5eaf-4051-888e-38fadaf902bb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-85ec9bcb-a863-4811-b6b7-4453e5f8c677 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:53.928640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:53.929876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:53.930903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:53.930903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:53.930903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:53.930903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:53.930903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:53.930903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:53.930903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:53.930903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:53.930903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:53.930903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:53.930903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:53.930903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:53.930903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:01:53.930903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:53.933369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 484/946] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:53.938719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec9bcb-a863-4811-b6b7-4453e5f8c677 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:53.950441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 463/947] 10.222.0.61 () {56 vars in 1149 bytes} [Thu Jul 14 10:01:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:53.953182 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f282037-5eaf-4051-888e-38fadaf902bb None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:54.059987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f282037-5eaf-4051-888e-38fadaf902bb None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:54.075611 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec9bcb-a863-4811-b6b7-4453e5f8c677 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:54.077948 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:54.113227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a557ed-d633-4f75-ad33-a637d097945c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:54.113900 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a557ed-d633-4f75-ad33-a637d097945c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:54.114183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a557ed-d633-4f75-ad33-a637d097945c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:54.114458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a557ed-d633-4f75-ad33-a637d097945c None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:54.115167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a557ed-d633-4f75-ad33-a637d097945c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:54.115472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a557ed-d633-4f75-ad33-a637d097945c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:54.116962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a557ed-d633-4f75-ad33-a637d097945c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:54.118258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde953cd-2b68-4324-a2ea-b01e47124d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:54.119442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde953cd-2b68-4324-a2ea-b01e47124d48 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:54.119724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde953cd-2b68-4324-a2ea-b01e47124d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:54.119995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde953cd-2b68-4324-a2ea-b01e47124d48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:54.122055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 485/948] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Jul 14 10:01:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:54.123253 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3877a-18ba-4f79-bbaf-edae30fa53f5 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:54.123543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3877a-18ba-4f79-bbaf-edae30fa53f5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:54.123819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e3877a-18ba-4f79-bbaf-edae30fa53f5 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:54.125254 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 486/949] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:54.127436 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:54.128441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3877a-18ba-4f79-bbaf-edae30fa53f5 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:54.133570 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde953cd-2b68-4324-a2ea-b01e47124d48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:54.135028 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde953cd-2b68-4324-a2ea-b01e47124d48 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:54.154499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-bde953cd-2b68-4324-a2ea-b01e47124d48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:54.154499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:54.154499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:54.154499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:54.154499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:54.154499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:54.154499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:54.154499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:54.154499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:54.154499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:54.154499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:54.154499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:54.154499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:54.154499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:54.154499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:54.155389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:54.156908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:54.156908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:54.156908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:54.156908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:54.162003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 464/950] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:54.163027 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733ccda-c709-4be1-b5ff-a6df8f528021 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:54.163318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733ccda-c709-4be1-b5ff-a6df8f528021 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:54.163591 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733ccda-c709-4be1-b5ff-a6df8f528021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:54.182575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4733ccda-c709-4be1-b5ff-a6df8f528021 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:54.184121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4733ccda-c709-4be1-b5ff-a6df8f528021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:54.235962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e3877a-18ba-4f79-bbaf-edae30fa53f5 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:54.266730 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 465/951] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:54.267590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733ccda-c709-4be1-b5ff-a6df8f528021 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:54.288475 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939e333-b0fb-433c-83b3-627696cf733e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:54.289581 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939e333-b0fb-433c-83b3-627696cf733e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:54.289865 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939e333-b0fb-433c-83b3-627696cf733e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:54.290137 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5939e333-b0fb-433c-83b3-627696cf733e None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:54.290882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939e333-b0fb-433c-83b3-627696cf733e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:54.291635 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939e333-b0fb-433c-83b3-627696cf733e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:54.293043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5939e333-b0fb-433c-83b3-627696cf733e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:54.297745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 487/952] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Jul 14 10:01:54 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:54.298644 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939e333-b0fb-433c-83b3-627696cf733e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:54.335415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:54.383777 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733ccda-c709-4be1-b5ff-a6df8f528021 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:54.407256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:54.419310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5939e333-b0fb-433c-83b3-627696cf733e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:54.422777 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a36079-b3c3-4f51-a1c4-f4c7f2c38250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:54.423536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a36079-b3c3-4f51-a1c4-f4c7f2c38250 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:54.423887 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a36079-b3c3-4f51-a1c4-f4c7f2c38250 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:54.424234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a36079-b3c3-4f51-a1c4-f4c7f2c38250 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:54.425100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a36079-b3c3-4f51-a1c4-f4c7f2c38250 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:54.425480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a36079-b3c3-4f51-a1c4-f4c7f2c38250 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:54.427221 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a36079-b3c3-4f51-a1c4-f4c7f2c38250 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:54.444869 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 466/953] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:54.446340 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a36079-b3c3-4f51-a1c4-f4c7f2c38250 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:54.468208 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc6d1f-f8bb-4936-963b-500445e178f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:54.469588 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc6d1f-f8bb-4936-963b-500445e178f8 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:54.469890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc6d1f-f8bb-4936-963b-500445e178f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:54.470221 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadc6d1f-f8bb-4936-963b-500445e178f8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:54.470895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc6d1f-f8bb-4936-963b-500445e178f8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:54.472637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadc6d1f-f8bb-4936-963b-500445e178f8 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-cadc6d1f-f8bb-4936-963b-500445e178f8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:54.487876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:54.488895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:54.489979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:54.489979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:54.489979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:54.489979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:54.489979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:54.489979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:54.489979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:54.489979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:54.489979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:54.489979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:54.489979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:54.489979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:54.489979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:01:54.489979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:54.490955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 488/954] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Jul 14 10:01:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:54.496472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc6d1f-f8bb-4936-963b-500445e178f8 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:54.597486 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a36079-b3c3-4f51-a1c4-f4c7f2c38250 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:54.607606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:54.618156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadc6d1f-f8bb-4936-963b-500445e178f8 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:54.639117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5ee9d-0b94-4d8c-93c4-73d77468b9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:54.639945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5ee9d-0b94-4d8c-93c4-73d77468b9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:54.640228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5ee9d-0b94-4d8c-93c4-73d77468b9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:54.640503 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5ee9d-0b94-4d8c-93c4-73d77468b9d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:54.647474 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5ee9d-0b94-4d8c-93c4-73d77468b9d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:54.649034 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e5ee9d-0b94-4d8c-93c4-73d77468b9d3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:54.660548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fec110-b8fb-4a8e-88a0-e9af48220290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:54.661625 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fec110-b8fb-4a8e-88a0-e9af48220290 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:54.662123 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fec110-b8fb-4a8e-88a0-e9af48220290 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:54.662612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fec110-b8fb-4a8e-88a0-e9af48220290 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:54.663268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-24e5ee9d-0b94-4d8c-93c4-73d77468b9d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:54.663268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:54.663268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:54.663268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:54.663268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:54.663268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:54.663268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:54.663268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:54.663268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:54.663268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:54.663268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:54.663268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:54.663268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:54.663268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:54.663268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:54.664177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:54.665120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:54.665120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:54.665120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:54.665120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:54.665120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:54.665120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:54.665120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:54.665120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:54.666696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fec110-b8fb-4a8e-88a0-e9af48220290 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:54.667223 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fec110-b8fb-4a8e-88a0-e9af48220290 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:54.668876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fec110-b8fb-4a8e-88a0-e9af48220290 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:54.669291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 467/955] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:54.670551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5ee9d-0b94-4d8c-93c4-73d77468b9d3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:54.675663 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 489/956] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Jul 14 10:01:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:54.686354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fec110-b8fb-4a8e-88a0-e9af48220290 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:54.724235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:54.788719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5ee9d-0b94-4d8c-93c4-73d77468b9d3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:54.796556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fec110-b8fb-4a8e-88a0-e9af48220290 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:54.828760 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38d4c4-3022-453b-b32d-4f18ba27126d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:54.829729 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38d4c4-3022-453b-b32d-4f18ba27126d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:54.830126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38d4c4-3022-453b-b32d-4f18ba27126d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:54.830722 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e38d4c4-3022-453b-b32d-4f18ba27126d None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:54.831963 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38d4c4-3022-453b-b32d-4f18ba27126d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:54.832538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38d4c4-3022-453b-b32d-4f18ba27126d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:54.834663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e38d4c4-3022-453b-b32d-4f18ba27126d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:54.843558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02381dbd-7ec1-46c9-85a3-351728abd6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:54.844867 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02381dbd-7ec1-46c9-85a3-351728abd6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:54.845235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02381dbd-7ec1-46c9-85a3-351728abd6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:54.845574 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02381dbd-7ec1-46c9-85a3-351728abd6f3 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:54.846886 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02381dbd-7ec1-46c9-85a3-351728abd6f3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:54.847531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02381dbd-7ec1-46c9-85a3-351728abd6f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:54.848964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02381dbd-7ec1-46c9-85a3-351728abd6f3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:54.853091 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 468/957] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:54.855491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38d4c4-3022-453b-b32d-4f18ba27126d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:54.863887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 490/958] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Jul 14 10:01:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:54.865898 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02381dbd-7ec1-46c9-85a3-351728abd6f3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:54.892806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:54.960565 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:54.964068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e38d4c4-3022-453b-b32d-4f18ba27126d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:54.975363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02381dbd-7ec1-46c9-85a3-351728abd6f3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:55.001496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b881f594-2e1e-4afd-8e47-c397d243e8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:55.002481 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881f594-2e1e-4afd-8e47-c397d243e8ac None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:55.002758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881f594-2e1e-4afd-8e47-c397d243e8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:55.003027 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b881f594-2e1e-4afd-8e47-c397d243e8ac None admin] PATH_INFO: `/v3/projects/57662256880f43018039715a755c087c/users/dd2b9ad7efa848938627385b411727ee/roles/9cb5d3eed6dc41589bac547cebf73cdb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:55.005757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881f594-2e1e-4afd-8e47-c397d243e8ac None admin] RBAC: Authorizing `identity:create_grant(project_id=57662256880f43018039715a755c087c, user_id=dd2b9ad7efa848938627385b411727ee, role_id=9cb5d3eed6dc41589bac547cebf73cdb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:55.006994 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b881f594-2e1e-4afd-8e47-c397d243e8ac None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:55.014308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae67231-0756-4f52-8267-2a761dfa46f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:55.015702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae67231-0756-4f52-8267-2a761dfa46f2 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:55.016010 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae67231-0756-4f52-8267-2a761dfa46f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:55.016379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae67231-0756-4f52-8267-2a761dfa46f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:55.017228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae67231-0756-4f52-8267-2a761dfa46f2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:55.018650 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae67231-0756-4f52-8267-2a761dfa46f2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:55.024411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 469/959] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:01:54 2022] PUT /identity/v3/projects/57662256880f43018039715a755c087c/users/dd2b9ad7efa848938627385b411727ee/roles/9cb5d3eed6dc41589bac547cebf73cdb => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:01:55.050184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 491/960] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Jul 14 10:01:54 2022] POST /identity/v3/endpoints => generated 349 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:01:55.301317 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0649dc-8e2c-43d2-aa05-94c74c3ce3fb None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:55.301664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0649dc-8e2c-43d2-aa05-94c74c3ce3fb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:55.301881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0649dc-8e2c-43d2-aa05-94c74c3ce3fb None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:55.303522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 470/961] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:55.308773 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08131f3a-6015-4a0b-a3be-1aa8ae291a74 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:55.309109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08131f3a-6015-4a0b-a3be-1aa8ae291a74 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:55.309382 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08131f3a-6015-4a0b-a3be-1aa8ae291a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:55.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-08131f3a-6015-4a0b-a3be-1aa8ae291a74 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:55.331315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-08131f3a-6015-4a0b-a3be-1aa8ae291a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:55.414910 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 492/962] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:55.440493 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0649dc-8e2c-43d2-aa05-94c74c3ce3fb None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:55.553359 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0649dc-8e2c-43d2-aa05-94c74c3ce3fb None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:55.589192 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa402161-cc08-45d1-8a82-2122fc93c58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:55.589863 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa402161-cc08-45d1-8a82-2122fc93c58e None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:55.590141 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa402161-cc08-45d1-8a82-2122fc93c58e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:55.590411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa402161-cc08-45d1-8a82-2122fc93c58e None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:55.591171 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa402161-cc08-45d1-8a82-2122fc93c58e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:55.592663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa402161-cc08-45d1-8a82-2122fc93c58e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:55.610388 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.sql.core [None req-aa402161-cc08-45d1-8a82-2122fc93c58e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 10:01:55.610388 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 10:01:55.610388 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [parameters: {'user_id': '791086359e5e4d61bc69c5b50147f1f8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 10:01:55.610388 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80320) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 14 10:01:55.618133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-aa402161-cc08-45d1-8a82-2122fc93c58e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 10:01:55.618133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:55.618133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:01:55.618133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:01:55.618133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:01:55.618133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:01:55.618133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:01:55.618133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:01:55.618133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:01:55.618133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:01:55.618133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:01:55.618133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:01:55.618133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:01:55.618133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result.read() Jul 14 10:01:55.618133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:01:55.619066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application yield resource Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application next(self.gen) Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.session.rollback() Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.force_reraise() Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise self.value Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application session.commit() Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 14 10:01:55.620095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._prepare_impl() Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.session.flush() Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._flush(objects) Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application compat.raise_( Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application flush_context.execute() Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rec.execute(self) Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 14 10:01:55.621725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 14 10:01:55.624036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result = self._query(query) Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application conn.query(q) Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application result.read() Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 14 10:01:55.626097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 14 10:01:55.628177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 14 10:01:55.628177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 14 10:01:55.628177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 14 10:01:55.628177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 14 10:01:55.628177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 14 10:01:55.628177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 14 10:01:55.628177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [parameters: {'user_id': '791086359e5e4d61bc69c5b50147f1f8', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 14 10:01:55.628177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 14 10:01:55.628177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:55.628177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:55.628177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:55.628177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:55.628177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:55.628177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:55.628177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 14 10:01:55.638345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 14 10:01:55.641202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 14 10:01:55.641202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 14 10:01:55.641202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 14 10:01:55.641202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 14 10:01:55.641202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 14 10:01:55.641202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:55.641202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 471/963] 10.222.0.61 () {60 vars in 1131 bytes} [Thu Jul 14 10:01:55 2022] POST /identity/v3/users => generated 160 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:55.642151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08131f3a-6015-4a0b-a3be-1aa8ae291a74 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:55.644769 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:55.756155 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08131f3a-6015-4a0b-a3be-1aa8ae291a74 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:55.791676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1684ccf-c3d1-42b3-8ace-0348f635f00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:55.792473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1684ccf-c3d1-42b3-8ace-0348f635f00f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:55.792748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1684ccf-c3d1-42b3-8ace-0348f635f00f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:55.793051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1684ccf-c3d1-42b3-8ace-0348f635f00f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:55.801683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1684ccf-c3d1-42b3-8ace-0348f635f00f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:55.803175 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1684ccf-c3d1-42b3-8ace-0348f635f00f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-d1684ccf-c3d1-42b3-8ace-0348f635f00f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:55.818130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:55.819114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:55.820101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:55.820101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:55.820101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:55.820101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:55.820101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:55.820101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:55.820101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:55.820101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:55.820101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 14 10:01:55.820101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:55.820101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 493/964] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:01:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:55.827442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa402161-cc08-45d1-8a82-2122fc93c58e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:55.857151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:55.936662 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa402161-cc08-45d1-8a82-2122fc93c58e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:55.972662 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945feaa7-9378-464e-9e4e-5ee41e67f3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:55.973363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945feaa7-9378-464e-9e4e-5ee41e67f3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:55.973645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945feaa7-9378-464e-9e4e-5ee41e67f3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:55.973919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945feaa7-9378-464e-9e4e-5ee41e67f3b8 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:55.974726 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945feaa7-9378-464e-9e4e-5ee41e67f3b8 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:55.975031 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945feaa7-9378-464e-9e4e-5ee41e67f3b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:55.976655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945feaa7-9378-464e-9e4e-5ee41e67f3b8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:56.016352 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 472/965] 10.222.0.61 () {56 vars in 1101 bytes} [Thu Jul 14 10:01:55 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:56.420607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133d8f4-5a8a-4ce2-960e-321176576ae9 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:56.421506 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133d8f4-5a8a-4ce2-960e-321176576ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:56.421705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133d8f4-5a8a-4ce2-960e-321176576ae9 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:56.423194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 494/966] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:56.428342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45c3cd-cfac-4680-adfb-341116f1e4bc None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:56.428634 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45c3cd-cfac-4680-adfb-341116f1e4bc None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:56.428904 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45c3cd-cfac-4680-adfb-341116f1e4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:56.447154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3a45c3cd-cfac-4680-adfb-341116f1e4bc None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:56.448633 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3a45c3cd-cfac-4680-adfb-341116f1e4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:56.517091 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 473/967] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:56 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:56.539707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133d8f4-5a8a-4ce2-960e-321176576ae9 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:56.601785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:56.640296 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133d8f4-5a8a-4ce2-960e-321176576ae9 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:56.674072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84adf50f-9c2c-43e0-b858-219084c7e272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:56.674736 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adf50f-9c2c-43e0-b858-219084c7e272 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:56.675017 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adf50f-9c2c-43e0-b858-219084c7e272 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:56.675285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84adf50f-9c2c-43e0-b858-219084c7e272 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:56.676272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adf50f-9c2c-43e0-b858-219084c7e272 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:56.677653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84adf50f-9c2c-43e0-b858-219084c7e272 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:56.692762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 495/968] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Jul 14 10:01:56 2022] POST /identity/v3/services => generated 242 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:01:57.138041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b732b3-234c-4c8f-a993-e27a1edffae6 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:57.138557 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b732b3-234c-4c8f-a993-e27a1edffae6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:57.138663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b732b3-234c-4c8f-a993-e27a1edffae6 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:57.140336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 474/969] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:57.145483 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f42baf-e837-45f9-80e0-d1ba80245d25 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:57.145882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f42baf-e837-45f9-80e0-d1ba80245d25 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:57.146044 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f42baf-e837-45f9-80e0-d1ba80245d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:57.162816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d5f42baf-e837-45f9-80e0-d1ba80245d25 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:57.164002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d5f42baf-e837-45f9-80e0-d1ba80245d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:57.235991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f0a51-6f80-4360-93a0-3682666265b3 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:57.236277 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f0a51-6f80-4360-93a0-3682666265b3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:57.236696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f0a51-6f80-4360-93a0-3682666265b3 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:57.238344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 475/970] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:01:57.241994 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 496/971] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:01:57.248081 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eadb4e-0af2-4fd7-b238-e7c08816f87e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:57.248374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eadb4e-0af2-4fd7-b238-e7c08816f87e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:57.248679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eadb4e-0af2-4fd7-b238-e7c08816f87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:57.265134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a8eadb4e-0af2-4fd7-b238-e7c08816f87e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:57.266675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a8eadb4e-0af2-4fd7-b238-e7c08816f87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:57.267237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f42baf-e837-45f9-80e0-d1ba80245d25 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:57.269596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:57.320732 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:57.336378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 476/972] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:01:57.359691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eadb4e-0af2-4fd7-b238-e7c08816f87e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:57.374015 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f42baf-e837-45f9-80e0-d1ba80245d25 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:57.409442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b587e-eb7a-44d5-a059-4473027ffb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:57.410254 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b587e-eb7a-44d5-a059-4473027ffb36 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:57.410535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b587e-eb7a-44d5-a059-4473027ffb36 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:57.410810 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b587e-eb7a-44d5-a059-4473027ffb36 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:57.422118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b587e-eb7a-44d5-a059-4473027ffb36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-0f8b587e-eb7a-44d5-a059-4473027ffb36 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:01:57.430538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:57.431498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:57.432363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:57.432363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:57.432363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:57.432363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:57.432363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:01:57.432363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:01:57.432363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:01:57.432363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:01:57.432363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:01:57.432363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:01:57.432363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:01:57.432363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:57.436542 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 497/973] 10.222.0.61 () {56 vars in 1120 bytes} [Thu Jul 14 10:01:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:57.437381 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b587e-eb7a-44d5-a059-4473027ffb36 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:57.469390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eadb4e-0af2-4fd7-b238-e7c08816f87e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:57.504655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5db170-4cef-4d8e-87e1-a37b55f9315c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:57.505488 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5db170-4cef-4d8e-87e1-a37b55f9315c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:57.505765 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5db170-4cef-4d8e-87e1-a37b55f9315c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:57.506029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5db170-4cef-4d8e-87e1-a37b55f9315c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:57.506679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5db170-4cef-4d8e-87e1-a37b55f9315c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:57.508041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5db170-4cef-4d8e-87e1-a37b55f9315c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-3c5db170-4cef-4d8e-87e1-a37b55f9315c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:57.520506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:57.521695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:57.522543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:57.522543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:57.522543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:57.522543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:57.522543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:57.522543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:57.522543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:57.522543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:57.522543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:57.522543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:57.522543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:57.522543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:57.522543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:01:57.522543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:57.522543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 477/974] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Jul 14 10:01:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:57.527837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5db170-4cef-4d8e-87e1-a37b55f9315c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:57.533235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:57.545795 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b587e-eb7a-44d5-a059-4473027ffb36 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:57.557636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:57.588579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564f01d-88b5-4f63-ad7b-70fde0fcc1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:57.589604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564f01d-88b5-4f63-ad7b-70fde0fcc1fd None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:57.589889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564f01d-88b5-4f63-ad7b-70fde0fcc1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:57.590419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564f01d-88b5-4f63-ad7b-70fde0fcc1fd None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:57.591612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564f01d-88b5-4f63-ad7b-70fde0fcc1fd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:57.591925 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564f01d-88b5-4f63-ad7b-70fde0fcc1fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:57.593982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5564f01d-88b5-4f63-ad7b-70fde0fcc1fd None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:57.602104 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 498/975] 10.222.0.61 () {56 vars in 1115 bytes} [Thu Jul 14 10:01:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:57.607571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564f01d-88b5-4f63-ad7b-70fde0fcc1fd None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:57.641465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5db170-4cef-4d8e-87e1-a37b55f9315c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:57.677508 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab14419-1dd4-4975-b2b4-2a6b1850f143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:57.678101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab14419-1dd4-4975-b2b4-2a6b1850f143 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:57.678379 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab14419-1dd4-4975-b2b4-2a6b1850f143 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:57.678659 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab14419-1dd4-4975-b2b4-2a6b1850f143 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:57.679363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab14419-1dd4-4975-b2b4-2a6b1850f143 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:57.679671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab14419-1dd4-4975-b2b4-2a6b1850f143 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:57.681056 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab14419-1dd4-4975-b2b4-2a6b1850f143 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:57.686055 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 478/976] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Jul 14 10:01:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:57.691703 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab14419-1dd4-4975-b2b4-2a6b1850f143 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:57.716407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564f01d-88b5-4f63-ad7b-70fde0fcc1fd None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:57.754659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffb820-58d2-43df-86ed-66b73876feb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:57.755468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffb820-58d2-43df-86ed-66b73876feb9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:57.755750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffb820-58d2-43df-86ed-66b73876feb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:57.756018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bffb820-58d2-43df-86ed-66b73876feb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:57.764990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:57.768094 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffb820-58d2-43df-86ed-66b73876feb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:57.769581 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bffb820-58d2-43df-86ed-66b73876feb9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:57.787771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-3bffb820-58d2-43df-86ed-66b73876feb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:57.787771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:57.787771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:57.787771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:57.787771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:57.787771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:57.787771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:57.787771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:57.787771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:57.787771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:57.787771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:57.787771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:57.787771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:57.787771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:57.787771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:57.788731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:57.789671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:57.789671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:57.789671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:57.789671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:57.794346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 499/977] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:57.795324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffb820-58d2-43df-86ed-66b73876feb9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:57.804962 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab14419-1dd4-4975-b2b4-2a6b1850f143 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:57.827393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:57.841340 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da037627-44b1-4366-9e6b-1617bfd8790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:57.842213 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da037627-44b1-4366-9e6b-1617bfd8790e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:57.843226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da037627-44b1-4366-9e6b-1617bfd8790e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:57.843643 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da037627-44b1-4366-9e6b-1617bfd8790e None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:57.844538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da037627-44b1-4366-9e6b-1617bfd8790e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:57.845010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da037627-44b1-4366-9e6b-1617bfd8790e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:57.846510 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da037627-44b1-4366-9e6b-1617bfd8790e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:57.852286 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 479/978] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Jul 14 10:01:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:57.857990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da037627-44b1-4366-9e6b-1617bfd8790e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:57.912451 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bffb820-58d2-43df-86ed-66b73876feb9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:57.947301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f30cf8-56af-4180-a570-1724882680fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:57.947900 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f30cf8-56af-4180-a570-1724882680fd None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:57.948175 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f30cf8-56af-4180-a570-1724882680fd None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:57.948442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f30cf8-56af-4180-a570-1724882680fd None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:57.949187 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f30cf8-56af-4180-a570-1724882680fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:57.949489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f30cf8-56af-4180-a570-1724882680fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:57.950839 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f30cf8-56af-4180-a570-1724882680fd None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:57.967728 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 500/979] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:57.968751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f30cf8-56af-4180-a570-1724882680fd None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:57.971033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da037627-44b1-4366-9e6b-1617bfd8790e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:58.003253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:58.009563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d64f6-1ff0-44d0-9fa1-a1a5de338ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:58.010228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d64f6-1ff0-44d0-9fa1-a1a5de338ebb None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:58.010512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d64f6-1ff0-44d0-9fa1-a1a5de338ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:58.010823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d64f6-1ff0-44d0-9fa1-a1a5de338ebb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:58.011637 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d64f6-1ff0-44d0-9fa1-a1a5de338ebb None admin] RBAC: Adding query filter params (interface=admin, service_id=01bba6c0a39e4b0e949a5f01b975ce5c, region_id=RegionOne) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:58.011951 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d64f6-1ff0-44d0-9fa1-a1a5de338ebb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:58.013345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d64f6-1ff0-44d0-9fa1-a1a5de338ebb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:58.019455 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 480/980] 10.222.0.61 () {56 vars in 1249 bytes} [Thu Jul 14 10:01:57 2022] GET /identity/v3/endpoints?service_id=01bba6c0a39e4b0e949a5f01b975ce5c&interface=admin®ion_id=RegionOne => generated 193 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:58.024715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d64f6-1ff0-44d0-9fa1-a1a5de338ebb None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:58.064947 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:58.083930 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f30cf8-56af-4180-a570-1724882680fd None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:58.120304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98a9eb-5cfc-4de9-825c-8ad7fa8d4450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:58.121418 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98a9eb-5cfc-4de9-825c-8ad7fa8d4450 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:58.121842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98a9eb-5cfc-4de9-825c-8ad7fa8d4450 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:58.122256 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b98a9eb-5cfc-4de9-825c-8ad7fa8d4450 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:58.129636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98a9eb-5cfc-4de9-825c-8ad7fa8d4450 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:58.131288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b98a9eb-5cfc-4de9-825c-8ad7fa8d4450 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:58.132675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d64f6-1ff0-44d0-9fa1-a1a5de338ebb None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-0b98a9eb-5cfc-4de9-825c-8ad7fa8d4450 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:01:58.146375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:58.147857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:58.149148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:58.149148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:58.149148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:01:58.149148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:01:58.149148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:01:58.149148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:01:58.149148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:01:58.149148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:01:58.149148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 14 10:01:58.149148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:58.152262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 501/981] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Jul 14 10:01:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:58.152977 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98a9eb-5cfc-4de9-825c-8ad7fa8d4450 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:58.173578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556e928-219e-4734-aa17-6d4de19c04df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:58.174249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556e928-219e-4734-aa17-6d4de19c04df None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:58.174521 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556e928-219e-4734-aa17-6d4de19c04df None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:58.174791 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556e928-219e-4734-aa17-6d4de19c04df None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:58.175732 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556e928-219e-4734-aa17-6d4de19c04df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:58.177162 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556e928-219e-4734-aa17-6d4de19c04df None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:58.183597 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 481/982] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Jul 14 10:01:58 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:01:58.275544 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b98a9eb-5cfc-4de9-825c-8ad7fa8d4450 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:58.313204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaeba5-fe6f-4f1f-a897-71968b2ca642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:58.313890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaeba5-fe6f-4f1f-a897-71968b2ca642 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:58.314173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaeba5-fe6f-4f1f-a897-71968b2ca642 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:58.314450 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaeba5-fe6f-4f1f-a897-71968b2ca642 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:58.315311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efaeba5-fe6f-4f1f-a897-71968b2ca642 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:58.315625 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efaeba5-fe6f-4f1f-a897-71968b2ca642 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:58.317109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efaeba5-fe6f-4f1f-a897-71968b2ca642 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:58.337215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 502/983] 10.222.0.61 () {56 vars in 1149 bytes} [Thu Jul 14 10:01:58 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:58.343269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556e928-219e-4734-aa17-6d4de19c04df None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:58.405818 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:58.448822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556e928-219e-4734-aa17-6d4de19c04df None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:58.483307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276a4300-d9e8-4f00-87ac-d5d150dd6e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:58.483958 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a4300-d9e8-4f00-87ac-d5d150dd6e79 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:58.484243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a4300-d9e8-4f00-87ac-d5d150dd6e79 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:58.484521 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276a4300-d9e8-4f00-87ac-d5d150dd6e79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:58.555272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a4300-d9e8-4f00-87ac-d5d150dd6e79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:58.556638 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276a4300-d9e8-4f00-87ac-d5d150dd6e79 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:58.614656 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:58.664467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 482/984] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:01:58 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:01:58.666098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaeba5-fe6f-4f1f-a897-71968b2ca642 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:58.668953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:58.733685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648c722-1659-43eb-b003-16850ed35c21 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:58.733967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648c722-1659-43eb-b003-16850ed35c21 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:58.734244 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b648c722-1659-43eb-b003-16850ed35c21 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:58.735625 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 483/985] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:01:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:01:58.740277 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31debcda-1b5e-4ef3-8fc1-a40691c2b52d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:58.740553 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31debcda-1b5e-4ef3-8fc1-a40691c2b52d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:58.740830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31debcda-1b5e-4ef3-8fc1-a40691c2b52d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:58.759359 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-31debcda-1b5e-4ef3-8fc1-a40691c2b52d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:01:58.760529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-31debcda-1b5e-4ef3-8fc1-a40691c2b52d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:01:58.771527 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaeba5-fe6f-4f1f-a897-71968b2ca642 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:58.807748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fd463-cbae-410f-8416-56a34abc1355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:58.808466 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fd463-cbae-410f-8416-56a34abc1355 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:58.808747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fd463-cbae-410f-8416-56a34abc1355 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:58.809049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fd463-cbae-410f-8416-56a34abc1355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:58.819802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fd463-cbae-410f-8416-56a34abc1355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:58.821262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fd463-cbae-410f-8416-56a34abc1355 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:58.832131 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 484/986] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:01:58 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:01:58.839387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-490fd463-cbae-410f-8416-56a34abc1355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:58.839387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:58.839387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:58.839387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:58.839387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:58.839387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:58.839387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:58.839387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:58.839387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:58.839387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:58.839387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:58.839387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:58.839387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:01:58.839387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:01:58.839387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:58.840507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:58.841635 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:01:58.841635 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:01:58.841635 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:01:58.841635 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:58.845242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 503/987] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:01:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:58.849937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31debcda-1b5e-4ef3-8fc1-a40691c2b52d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:58.855133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fd463-cbae-410f-8416-56a34abc1355 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:58.913287 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:58.944628 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:58.956560 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31debcda-1b5e-4ef3-8fc1-a40691c2b52d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:58.960381 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fd463-cbae-410f-8416-56a34abc1355 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:58.993274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad73fabc-b9d4-4309-b5ea-8dc35186c042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:58.993932 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73fabc-b9d4-4309-b5ea-8dc35186c042 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:58.994211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73fabc-b9d4-4309-b5ea-8dc35186c042 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:58.994482 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73fabc-b9d4-4309-b5ea-8dc35186c042 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:58.995202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad73fabc-b9d4-4309-b5ea-8dc35186c042 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:58.995525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad73fabc-b9d4-4309-b5ea-8dc35186c042 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:58.996864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad73fabc-b9d4-4309-b5ea-8dc35186c042 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:58.998892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcc92a-9f51-4451-9862-9ef33b39d65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:58.999913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcc92a-9f51-4451-9862-9ef33b39d65d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:59.000260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcc92a-9f51-4451-9862-9ef33b39d65d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:59.000539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdcc92a-9f51-4451-9862-9ef33b39d65d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:59.001213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdcc92a-9f51-4451-9862-9ef33b39d65d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:59.002557 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdcc92a-9f51-4451-9862-9ef33b39d65d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:59.013641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 485/988] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:59.015252 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad73fabc-b9d4-4309-b5ea-8dc35186c042 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-ccdcc92a-9f51-4451-9862-9ef33b39d65d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:59.015385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:59.016330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:59.017261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:59.017261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:59.017261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:59.017261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:59.017261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:59.017261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:59.017261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:59.017261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:59.017261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:59.017261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:59.017261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:01:59.017261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:01:59.017261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:01:59.017261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:01:59.017261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 504/989] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:01:59.023105 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcc92a-9f51-4451-9862-9ef33b39d65d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:59.124224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad73fabc-b9d4-4309-b5ea-8dc35186c042 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:59.130538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdcc92a-9f51-4451-9862-9ef33b39d65d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:59.148449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:59.159524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ab209-bc34-4fbe-a904-a5e69c7ace8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:59.160314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ab209-bc34-4fbe-a904-a5e69c7ace8d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:59.160593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ab209-bc34-4fbe-a904-a5e69c7ace8d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:59.160864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785ab209-bc34-4fbe-a904-a5e69c7ace8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:59.166600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994cb52-587b-467f-8667-f168e7fb40ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:59.167254 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994cb52-587b-467f-8667-f168e7fb40ce None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:59.167534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994cb52-587b-467f-8667-f168e7fb40ce None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:59.167805 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994cb52-587b-467f-8667-f168e7fb40ce None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:59.168145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ab209-bc34-4fbe-a904-a5e69c7ace8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:59.168492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994cb52-587b-467f-8667-f168e7fb40ce None admin] RBAC: Adding query filter params (name=compute) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:59.168853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994cb52-587b-467f-8667-f168e7fb40ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:59.169682 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785ab209-bc34-4fbe-a904-a5e69c7ace8d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:59.170798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994cb52-587b-467f-8667-f168e7fb40ce None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:59.176023 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 505/990] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Jul 14 10:01:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:01:59.181267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994cb52-587b-467f-8667-f168e7fb40ce None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:59.184835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-785ab209-bc34-4fbe-a904-a5e69c7ace8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:59.184835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:01:59.184835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:01:59.184835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:01:59.184835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:01:59.184835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:01:59.184835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:01:59.184835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:01:59.184835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:01:59.184835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:01:59.184835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:01:59.184835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:01:59.184835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:01:59.184835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:01:59.184835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:01:59.186770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:01:59.187680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:01:59.187680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:01:59.187680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:01:59.187680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:01:59.187680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:01:59.187680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:01:59.187680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:01:59.187680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:01:59.187680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 486/991] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:01:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:01:59.192025 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ab209-bc34-4fbe-a904-a5e69c7ace8d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:59.203771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:59.295865 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994cb52-587b-467f-8667-f168e7fb40ce None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:59.315444 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785ab209-bc34-4fbe-a904-a5e69c7ace8d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:59.333471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45eae4-851b-48de-bcfe-84a20eb769d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:59.334132 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45eae4-851b-48de-bcfe-84a20eb769d7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:59.334411 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45eae4-851b-48de-bcfe-84a20eb769d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:59.334682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e45eae4-851b-48de-bcfe-84a20eb769d7 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:59.335387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45eae4-851b-48de-bcfe-84a20eb769d7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:59.335686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45eae4-851b-48de-bcfe-84a20eb769d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:59.337079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e45eae4-851b-48de-bcfe-84a20eb769d7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:59.341998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 506/992] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Jul 14 10:01:59 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:59.347721 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45eae4-851b-48de-bcfe-84a20eb769d7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:59.349935 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217b1c9-bf68-4551-8332-cbf7a37108ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:59.350598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217b1c9-bf68-4551-8332-cbf7a37108ae None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:59.350879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217b1c9-bf68-4551-8332-cbf7a37108ae None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:59.351151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2217b1c9-bf68-4551-8332-cbf7a37108ae None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:59.351943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217b1c9-bf68-4551-8332-cbf7a37108ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:59.352248 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217b1c9-bf68-4551-8332-cbf7a37108ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:59.353687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2217b1c9-bf68-4551-8332-cbf7a37108ae None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:59.371027 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 487/993] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:01:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:59.372016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217b1c9-bf68-4551-8332-cbf7a37108ae None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:59.399946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:59.443752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:59.455175 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e45eae4-851b-48de-bcfe-84a20eb769d7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:59.476013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2217b1c9-bf68-4551-8332-cbf7a37108ae None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:59.490821 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a012b-1160-4ee7-878e-e51401886e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:59.491803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a012b-1160-4ee7-878e-e51401886e45 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:59.492091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a012b-1160-4ee7-878e-e51401886e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:59.492521 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a012b-1160-4ee7-878e-e51401886e45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:59.493635 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a012b-1160-4ee7-878e-e51401886e45 None admin] RBAC: Adding query filter params (interface=public, service_id=448bce1db3b44198ad7d825d066567ca, region_id=RegionOne) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:59.493950 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a012b-1160-4ee7-878e-e51401886e45 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:59.495309 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a012b-1160-4ee7-878e-e51401886e45 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:59.506228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 507/994] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Jul 14 10:01:59 2022] GET /identity/v3/endpoints?service_id=448bce1db3b44198ad7d825d066567ca&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:59.507492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a012b-1160-4ee7-878e-e51401886e45 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:01:59.510665 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ae6408-f459-4030-b594-a97ef12b495f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:59.511308 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae6408-f459-4030-b594-a97ef12b495f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:59.511588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae6408-f459-4030-b594-a97ef12b495f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:59.511859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ae6408-f459-4030-b594-a97ef12b495f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:59.512685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae6408-f459-4030-b594-a97ef12b495f None admin] RBAC: Adding query filter params (role.id=9cb5d3eed6dc41589bac547cebf73cdb, scope.project.id=57662256880f43018039715a755c087c, user.id=dd2b9ad7efa848938627385b411727ee) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:01:59.513026 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae6408-f459-4030-b594-a97ef12b495f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:59.514541 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ae6408-f459-4030-b594-a97ef12b495f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:59.521280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 488/995] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:01:59 2022] GET /identity/v3/role_assignments?user.id=dd2b9ad7efa848938627385b411727ee&scope.project.id=57662256880f43018039715a755c087c&role.id=9cb5d3eed6dc41589bac547cebf73cdb => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:01:59.622919 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a012b-1160-4ee7-878e-e51401886e45 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:01:59.634627 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:01:59.660882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565b750-6fba-4c9e-887b-c9f5c2462450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:01:59.661972 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565b750-6fba-4c9e-887b-c9f5c2462450 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:01:59.662459 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565b750-6fba-4c9e-887b-c9f5c2462450 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:01:59.663019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565b750-6fba-4c9e-887b-c9f5c2462450 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:01:59.663912 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565b750-6fba-4c9e-887b-c9f5c2462450 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:01:59.665508 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565b750-6fba-4c9e-887b-c9f5c2462450 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:01:59.673603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 508/996] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Jul 14 10:01:59 2022] GET /identity/v3/services => generated 1727 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:02:00.251859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a177f3-e17d-473e-82f4-586f9e3cc837 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:00.252280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a177f3-e17d-473e-82f4-586f9e3cc837 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:00.252470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a177f3-e17d-473e-82f4-586f9e3cc837 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:00.253840 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 489/997] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:00.259167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da1d7b-2b6e-42d5-83a7-042ad34e964a None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:00.259578 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da1d7b-2b6e-42d5-83a7-042ad34e964a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:00.259749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da1d7b-2b6e-42d5-83a7-042ad34e964a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:00.277743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a1da1d7b-2b6e-42d5-83a7-042ad34e964a None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:00.278934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a1da1d7b-2b6e-42d5-83a7-042ad34e964a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:00.346259 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 509/998] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:00 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:00.368782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a177f3-e17d-473e-82f4-586f9e3cc837 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:00.471341 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a177f3-e17d-473e-82f4-586f9e3cc837 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:00.504808 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce7292-50aa-4ead-8279-25fb3ef33bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:00.505578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce7292-50aa-4ead-8279-25fb3ef33bbd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:00.505862 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce7292-50aa-4ead-8279-25fb3ef33bbd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:00.506138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce7292-50aa-4ead-8279-25fb3ef33bbd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:00.506789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce7292-50aa-4ead-8279-25fb3ef33bbd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:00.508143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce7292-50aa-4ead-8279-25fb3ef33bbd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:00.509924 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e7ce7292-50aa-4ead-8279-25fb3ef33bbd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:00.522336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:00.523325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:00.524185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:00.524185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:00.524185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:00.524185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:00.524185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:00.524185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:00.524185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:00.524185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:00.524185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:00.524185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:00.524185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:02:00.524185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:02:00.524185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 14 10:02:00.524185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:00.528594 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 490/999] 10.222.0.61 () {56 vars in 1126 bytes} [Thu Jul 14 10:02:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:00.530177 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da1d7b-2b6e-42d5-83a7-042ad34e964a None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:00.632293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da1d7b-2b6e-42d5-83a7-042ad34e964a None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:00.634641 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:00.667724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec054122-4d45-44b0-8671-f8e24befcafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:00.668395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec054122-4d45-44b0-8671-f8e24befcafe None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:00.668678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec054122-4d45-44b0-8671-f8e24befcafe None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:00.668979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec054122-4d45-44b0-8671-f8e24befcafe None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:00.670416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec054122-4d45-44b0-8671-f8e24befcafe None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:00.670806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec054122-4d45-44b0-8671-f8e24befcafe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:00.672208 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec054122-4d45-44b0-8671-f8e24befcafe None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:00.680995 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 510/1000] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Jul 14 10:02:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:00.683517 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce7292-50aa-4ead-8279-25fb3ef33bbd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:00.798212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce7292-50aa-4ead-8279-25fb3ef33bbd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:00.832081 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a2241-eec5-44ae-b8a3-3090296d69a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:00.832739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a2241-eec5-44ae-b8a3-3090296d69a9 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:00.833043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a2241-eec5-44ae-b8a3-3090296d69a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:00.833320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a2241-eec5-44ae-b8a3-3090296d69a9 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:00.834016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a2241-eec5-44ae-b8a3-3090296d69a9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:00.834321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a2241-eec5-44ae-b8a3-3090296d69a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:00.835743 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a2241-eec5-44ae-b8a3-3090296d69a9 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:00.845712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 491/1001] 10.222.0.61 () {56 vars in 1122 bytes} [Thu Jul 14 10:02:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:00.847201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec054122-4d45-44b0-8671-f8e24befcafe None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:00.947611 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec054122-4d45-44b0-8671-f8e24befcafe None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:00.980644 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d31980-7c7c-4c13-bf44-527b3d738d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:00.981327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d31980-7c7c-4c13-bf44-527b3d738d7c None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:00.981618 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d31980-7c7c-4c13-bf44-527b3d738d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:00.981895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d31980-7c7c-4c13-bf44-527b3d738d7c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:00.982677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d31980-7c7c-4c13-bf44-527b3d738d7c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:00.984032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d31980-7c7c-4c13-bf44-527b3d738d7c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:01.008960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 511/1002] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Jul 14 10:02:00 2022] POST /identity/v3/endpoints => generated 325 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:02:01.623854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed778215-ca21-46bd-b185-5154a1b3ba14 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:01.624363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed778215-ca21-46bd-b185-5154a1b3ba14 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:01.624363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed778215-ca21-46bd-b185-5154a1b3ba14 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:01.625852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 492/1003] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:01.630765 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd203fc7-1be4-4c85-94a7-1acff6c96bbc None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:01.631144 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd203fc7-1be4-4c85-94a7-1acff6c96bbc None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:01.631311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd203fc7-1be4-4c85-94a7-1acff6c96bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:01.648531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-fd203fc7-1be4-4c85-94a7-1acff6c96bbc None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:01.649742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fd203fc7-1be4-4c85-94a7-1acff6c96bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:01.673030 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:01.711877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cefdad-b7ef-46f1-b036-a0766f2ea935 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:01.712162 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cefdad-b7ef-46f1-b036-a0766f2ea935 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:01.712435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cefdad-b7ef-46f1-b036-a0766f2ea935 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:01.718011 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 493/1004] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:01.719246 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc9704-435e-4003-8f70-6e7565306675 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:01.719591 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc9704-435e-4003-8f70-6e7565306675 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:01.719931 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc9704-435e-4003-8f70-6e7565306675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:01.730365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 512/1005] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:01 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:01.737050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d7dc9704-435e-4003-8f70-6e7565306675 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:01.738239 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d7dc9704-435e-4003-8f70-6e7565306675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:01.753803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd203fc7-1be4-4c85-94a7-1acff6c96bbc None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:01.800723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:01.813816 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 494/1006] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:01 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:01.835982 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc9704-435e-4003-8f70-6e7565306675 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:01.862965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd203fc7-1be4-4c85-94a7-1acff6c96bbc None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:01.898246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e2ab8-ab97-4505-b699-c55596454641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:01.898967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e2ab8-ab97-4505-b699-c55596454641 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:01.899244 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e2ab8-ab97-4505-b699-c55596454641 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:01.899516 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e2ab8-ab97-4505-b699-c55596454641 None admin] PATH_INFO: `/v3/services/image` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:01.900158 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e2ab8-ab97-4505-b699-c55596454641 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:01.901558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e2ab8-ab97-4505-b699-c55596454641 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-602e2ab8-ab97-4505-b699-c55596454641 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:01.914314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:01.915323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:01.917871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:01.917871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:01.917871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:01.917871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:01.917871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:01.917871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:01.917871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:01.917871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:01.917871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:01.917871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:01.917871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:02:01.917871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:02:01.917871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:02:01.917871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:01.917871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 513/1007] 10.222.0.61 () {56 vars in 1105 bytes} [Thu Jul 14 10:02:01 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:02:01.921850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e2ab8-ab97-4505-b699-c55596454641 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:01.941803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:01.944364 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc9704-435e-4003-8f70-6e7565306675 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:01.980594 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa496e-f67b-4187-8e8d-1266e7fbd92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:01.981564 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa496e-f67b-4187-8e8d-1266e7fbd92c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:01.981841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa496e-f67b-4187-8e8d-1266e7fbd92c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:01.982137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa496e-f67b-4187-8e8d-1266e7fbd92c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:01.983365 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa496e-f67b-4187-8e8d-1266e7fbd92c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:01.984713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa496e-f67b-4187-8e8d-1266e7fbd92c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-15fa496e-f67b-4187-8e8d-1266e7fbd92c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:01.997442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:01.998485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:01.999370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:01.999370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:01.999370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:01.999370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:01.999370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:01.999370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:01.999370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:01.999370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:01.999370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:01.999370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:01.999370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:02:01.999370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:02:01.999370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 14 10:02:01.999370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:01.999370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 495/1008] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:02:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:02:02.004596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa496e-f67b-4187-8e8d-1266e7fbd92c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:02.033429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e2ab8-ab97-4505-b699-c55596454641 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:02.068952 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:02.069732 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6c00d-5319-48c3-b6c5-0995f738fb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:02.070567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c00d-5319-48c3-b6c5-0995f738fb71 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:02.070681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c00d-5319-48c3-b6c5-0995f738fb71 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:02.071083 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb6c00d-5319-48c3-b6c5-0995f738fb71 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:02.071662 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6c00d-5319-48c3-b6c5-0995f738fb71 None admin] RBAC: Adding query filter params (name=image) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:02.072015 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6c00d-5319-48c3-b6c5-0995f738fb71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:02.073438 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb6c00d-5319-48c3-b6c5-0995f738fb71 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:02.083786 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 514/1009] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:01 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:02.084773 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6c00d-5319-48c3-b6c5-0995f738fb71 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:02.119858 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa496e-f67b-4187-8e8d-1266e7fbd92c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:02.160114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338eea1b-f420-428c-a20d-9029feb4e045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:02.160776 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338eea1b-f420-428c-a20d-9029feb4e045 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:02.161098 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338eea1b-f420-428c-a20d-9029feb4e045 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:02.161370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338eea1b-f420-428c-a20d-9029feb4e045 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:02.162078 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338eea1b-f420-428c-a20d-9029feb4e045 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:02.162381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338eea1b-f420-428c-a20d-9029feb4e045 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:02.163732 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338eea1b-f420-428c-a20d-9029feb4e045 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:02.168909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 496/1010] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Jul 14 10:02:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:02.174237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338eea1b-f420-428c-a20d-9029feb4e045 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:02.176106 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:02.196473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb6c00d-5319-48c3-b6c5-0995f738fb71 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:02.241619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ec6dc-7c63-4859-a851-44e683be0b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:02.242291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ec6dc-7c63-4859-a851-44e683be0b55 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:02.242571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ec6dc-7c63-4859-a851-44e683be0b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:02.242844 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ec6dc-7c63-4859-a851-44e683be0b55 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:02.243552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ec6dc-7c63-4859-a851-44e683be0b55 None admin] RBAC: Adding query filter params (type=image) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:02.243858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ec6dc-7c63-4859-a851-44e683be0b55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:02.245263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ec6dc-7c63-4859-a851-44e683be0b55 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:02.261202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 515/1011] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:02 2022] GET /identity/v3/services?type=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:02.296879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338eea1b-f420-428c-a20d-9029feb4e045 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:02.319781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:02.336180 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb224771-1e42-4446-8dbb-28cd76907b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:02.336859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb224771-1e42-4446-8dbb-28cd76907b4e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:02.337164 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb224771-1e42-4446-8dbb-28cd76907b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:02.337439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb224771-1e42-4446-8dbb-28cd76907b4e None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:02.338167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb224771-1e42-4446-8dbb-28cd76907b4e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:02.338473 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb224771-1e42-4446-8dbb-28cd76907b4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:02.339844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb224771-1e42-4446-8dbb-28cd76907b4e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:02.351186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 497/1012] 10.222.0.61 () {56 vars in 1112 bytes} [Thu Jul 14 10:02:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:02.352505 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ec6dc-7c63-4859-a851-44e683be0b55 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:02.458837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ec6dc-7c63-4859-a851-44e683be0b55 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:02.492690 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61c5dd-8ba0-4fcd-a1f8-e0958a1d172b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:02.493383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61c5dd-8ba0-4fcd-a1f8-e0958a1d172b None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:02.493664 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61c5dd-8ba0-4fcd-a1f8-e0958a1d172b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:02.493939 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61c5dd-8ba0-4fcd-a1f8-e0958a1d172b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:02.494675 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61c5dd-8ba0-4fcd-a1f8-e0958a1d172b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:02.496136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61c5dd-8ba0-4fcd-a1f8-e0958a1d172b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:02.515740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 516/1013] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Jul 14 10:02:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:02:03.049961 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79d5c8-7dac-4048-ad89-b68b95e3ff47 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:03.050482 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79d5c8-7dac-4048-ad89-b68b95e3ff47 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:03.050598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d79d5c8-7dac-4048-ad89-b68b95e3ff47 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:03.052030 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 498/1014] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:03.057117 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316319a6-0999-4d68-b280-996b569669a0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:03.057524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316319a6-0999-4d68-b280-996b569669a0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:03.057741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316319a6-0999-4d68-b280-996b569669a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:03.062816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:03.075176 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-316319a6-0999-4d68-b280-996b569669a0 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:03.077068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-316319a6-0999-4d68-b280-996b569669a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:03.149182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 517/1015] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:03 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:03.172985 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79d5c8-7dac-4048-ad89-b68b95e3ff47 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:03.280221 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d79d5c8-7dac-4048-ad89-b68b95e3ff47 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:03.316001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558064be-3e91-40ea-8e71-90b6dd0a4248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:03.316716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558064be-3e91-40ea-8e71-90b6dd0a4248 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:03.317014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558064be-3e91-40ea-8e71-90b6dd0a4248 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:03.317281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558064be-3e91-40ea-8e71-90b6dd0a4248 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:03.318452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558064be-3e91-40ea-8e71-90b6dd0a4248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:03.319868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558064be-3e91-40ea-8e71-90b6dd0a4248 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:03.326314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 499/1016] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:03.328379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316319a6-0999-4d68-b280-996b569669a0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:03.436527 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316319a6-0999-4d68-b280-996b569669a0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:03.464581 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:03.473427 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea89f7-db1a-4dd3-82d1-547cc257b79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:03.474092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea89f7-db1a-4dd3-82d1-547cc257b79f None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:03.474375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea89f7-db1a-4dd3-82d1-547cc257b79f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:03.474653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea89f7-db1a-4dd3-82d1-547cc257b79f None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:03.475368 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea89f7-db1a-4dd3-82d1-547cc257b79f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:03.476794 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea89f7-db1a-4dd3-82d1-547cc257b79f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:03.504372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 518/1017] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Jul 14 10:02:03 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:02:04.297083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324a2ce-0a65-4fea-a20f-31684cb414b5 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:04.297592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324a2ce-0a65-4fea-a20f-31684cb414b5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:04.297698 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324a2ce-0a65-4fea-a20f-31684cb414b5 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:04.299151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 500/1018] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:04.304270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b78fb-a18a-4878-8827-0064f1d549a7 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:04.304650 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b78fb-a18a-4878-8827-0064f1d549a7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:04.304808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b78fb-a18a-4878-8827-0064f1d549a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:04.321394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3c6b78fb-a18a-4878-8827-0064f1d549a7 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:04.322556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3c6b78fb-a18a-4878-8827-0064f1d549a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:04.390861 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 519/1019] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:04 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:04.412623 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324a2ce-0a65-4fea-a20f-31684cb414b5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:04.467843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:04.517393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324a2ce-0a65-4fea-a20f-31684cb414b5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:04.552274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91bc4e-7216-4ba0-8550-bf78060e93dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:04.552950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91bc4e-7216-4ba0-8550-bf78060e93dc None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:04.553239 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91bc4e-7216-4ba0-8550-bf78060e93dc None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:04.553509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91bc4e-7216-4ba0-8550-bf78060e93dc None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:04.554222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91bc4e-7216-4ba0-8550-bf78060e93dc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:04.555618 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91bc4e-7216-4ba0-8550-bf78060e93dc None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:04.577092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 501/1020] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Jul 14 10:02:04 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:02:04.662630 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367d4e0-c591-4ae8-b296-b8f610ae51b1 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:04.662984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367d4e0-c591-4ae8-b296-b8f610ae51b1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:04.663186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a367d4e0-c591-4ae8-b296-b8f610ae51b1 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:04.664754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 520/1021] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:04.670645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da10c3-a039-4f6c-85f0-ae03904e4a9b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:04.670935 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da10c3-a039-4f6c-85f0-ae03904e4a9b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:04.671212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da10c3-a039-4f6c-85f0-ae03904e4a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:04.690390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a0da10c3-a039-4f6c-85f0-ae03904e4a9b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:04.691628 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a0da10c3-a039-4f6c-85f0-ae03904e4a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:04.769099 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 502/1022] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:04.792556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367d4e0-c591-4ae8-b296-b8f610ae51b1 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:04.895528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a367d4e0-c591-4ae8-b296-b8f610ae51b1 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:04.906672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:04.931908 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d6cfe-e526-4dda-bd1b-f7529b37b5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:04.932712 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6cfe-e526-4dda-bd1b-f7529b37b5bc None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:04.933012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6cfe-e526-4dda-bd1b-f7529b37b5bc None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:04.933285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6cfe-e526-4dda-bd1b-f7529b37b5bc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:04.943245 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d6cfe-e526-4dda-bd1b-f7529b37b5bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-669d6cfe-e526-4dda-bd1b-f7529b37b5bc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:02:04.951367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:04.952337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:04.953255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:04.953255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:04.953255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:04.953255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:04.953255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:02:04.953255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:02:04.953255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:02:04.953255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:02:04.953255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:02:04.953255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:02:04.953255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:02:04.953255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:04.953255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 521/1023] 10.222.0.61 () {56 vars in 1120 bytes} [Thu Jul 14 10:02:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:02:04.960234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da10c3-a039-4f6c-85f0-ae03904e4a9b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:05.002900 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:05.065877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da10c3-a039-4f6c-85f0-ae03904e4a9b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:05.100266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a706f-1357-4484-9b69-d41d452c5db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:05.100949 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a706f-1357-4484-9b69-d41d452c5db0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:05.101248 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a706f-1357-4484-9b69-d41d452c5db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:05.101520 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a706f-1357-4484-9b69-d41d452c5db0 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:05.102272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a706f-1357-4484-9b69-d41d452c5db0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:05.102579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a706f-1357-4484-9b69-d41d452c5db0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:05.103939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a706f-1357-4484-9b69-d41d452c5db0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:05.116766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 503/1024] 10.222.0.61 () {56 vars in 1115 bytes} [Thu Jul 14 10:02:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:05.118109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d6cfe-e526-4dda-bd1b-f7529b37b5bc None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:05.220193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d6cfe-e526-4dda-bd1b-f7529b37b5bc None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:05.260428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea61bf-45d6-40d6-adf2-eadab7ef9ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:05.261261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea61bf-45d6-40d6-adf2-eadab7ef9ffc None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:05.261543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea61bf-45d6-40d6-adf2-eadab7ef9ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:05.261816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ea61bf-45d6-40d6-adf2-eadab7ef9ffc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:05.273440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea61bf-45d6-40d6-adf2-eadab7ef9ffc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:05.274895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ea61bf-45d6-40d6-adf2-eadab7ef9ffc None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:05.293398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-00ea61bf-45d6-40d6-adf2-eadab7ef9ffc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:02:05.293398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:05.293398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:05.293398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:05.293398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:05.293398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:05.293398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:05.293398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:05.293398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:05.293398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:05.293398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:05.293398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:05.293398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:02:05.293398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:02:05.293398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:05.294393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:05.295319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:02:05.295319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:02:05.295319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:02:05.295319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:05.295319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 522/1025] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:05.302195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a706f-1357-4484-9b69-d41d452c5db0 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:05.401077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:05.405452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a706f-1357-4484-9b69-d41d452c5db0 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:05.439577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e055588-e4d9-47b9-9da0-025035559d41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:05.440243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e055588-e4d9-47b9-9da0-025035559d41 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:05.440524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e055588-e4d9-47b9-9da0-025035559d41 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:05.440797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e055588-e4d9-47b9-9da0-025035559d41 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:05.441565 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e055588-e4d9-47b9-9da0-025035559d41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:05.441872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e055588-e4d9-47b9-9da0-025035559d41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:05.443228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e055588-e4d9-47b9-9da0-025035559d41 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:05.460057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 504/1026] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:05.463822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea61bf-45d6-40d6-adf2-eadab7ef9ffc None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:05.486450 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:05.562459 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ea61bf-45d6-40d6-adf2-eadab7ef9ffc None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:05.588860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045252c-8270-41a2-89c1-bda5c32e77d3 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:05.589184 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045252c-8270-41a2-89c1-bda5c32e77d3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:05.589440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c045252c-8270-41a2-89c1-bda5c32e77d3 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:05.590999 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 505/1027] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:05.595622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ded914-c786-428e-abbe-ce46e1aa8260 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:05.595902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ded914-c786-428e-abbe-ce46e1aa8260 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:05.596171 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ded914-c786-428e-abbe-ce46e1aa8260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:05.597997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5355e9-05cf-4479-afcb-706a07392c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:05.598793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5355e9-05cf-4479-afcb-706a07392c0e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:05.599076 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5355e9-05cf-4479-afcb-706a07392c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:05.599350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5355e9-05cf-4479-afcb-706a07392c0e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:05.607120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5355e9-05cf-4479-afcb-706a07392c0e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:05.608550 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5355e9-05cf-4479-afcb-706a07392c0e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:05.612666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b7ded914-c786-428e-abbe-ce46e1aa8260 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:05.613876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b7ded914-c786-428e-abbe-ce46e1aa8260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-fa5355e9-05cf-4479-afcb-706a07392c0e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:02:05.623567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:05.624718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:05.626204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:05.626204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:05.626204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:02:05.626204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:02:05.626204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:02:05.626204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:02:05.626204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:02:05.626204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:02:05.626204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:02:05.626204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:05.626204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 523/1028] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Jul 14 10:02:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:02:05.631220 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5355e9-05cf-4479-afcb-706a07392c0e None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:05.689648 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 506/1029] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:02:05.714487 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ded914-c786-428e-abbe-ce46e1aa8260 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:05.719457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:05.739511 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5355e9-05cf-4479-afcb-706a07392c0e None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:05.773902 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da624e-cea3-4522-a879-e0dca991dbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:05.774576 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da624e-cea3-4522-a879-e0dca991dbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:05.774909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da624e-cea3-4522-a879-e0dca991dbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:05.775171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47da624e-cea3-4522-a879-e0dca991dbe6 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:05.776043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da624e-cea3-4522-a879-e0dca991dbe6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:05.776348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da624e-cea3-4522-a879-e0dca991dbe6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:05.777800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47da624e-cea3-4522-a879-e0dca991dbe6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:05.796954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 524/1030] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Jul 14 10:02:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:05.802542 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da624e-cea3-4522-a879-e0dca991dbe6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:05.813020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:05.823548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ded914-c786-428e-abbe-ce46e1aa8260 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:05.861116 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adf640-22a5-4f87-a51d-fa941baf3ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:05.861909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adf640-22a5-4f87-a51d-fa941baf3ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:05.862190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adf640-22a5-4f87-a51d-fa941baf3ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:05.862592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adf640-22a5-4f87-a51d-fa941baf3ee1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:05.863713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adf640-22a5-4f87-a51d-fa941baf3ee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:05.865145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adf640-22a5-4f87-a51d-fa941baf3ee1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:05.866769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 507/1031] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:05.872218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adf640-22a5-4f87-a51d-fa941baf3ee1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:05.909468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47da624e-cea3-4522-a879-e0dca991dbe6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:05.944114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8858aaf-c3e7-4d2d-9472-d3af22d98715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:05.944692 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8858aaf-c3e7-4d2d-9472-d3af22d98715 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:05.944995 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8858aaf-c3e7-4d2d-9472-d3af22d98715 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:05.945268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8858aaf-c3e7-4d2d-9472-d3af22d98715 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:05.955945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:05.976723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adf640-22a5-4f87-a51d-fa941baf3ee1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:06.016089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de24f2-5c56-4991-a010-0b9e8a4a1429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:06.016753 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de24f2-5c56-4991-a010-0b9e8a4a1429 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:06.017549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de24f2-5c56-4991-a010-0b9e8a4a1429 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:06.017823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de24f2-5c56-4991-a010-0b9e8a4a1429 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:06.018527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de24f2-5c56-4991-a010-0b9e8a4a1429 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:06.019127 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8858aaf-c3e7-4d2d-9472-d3af22d98715 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:06.019940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58de24f2-5c56-4991-a010-0b9e8a4a1429 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:06.020558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8858aaf-c3e7-4d2d-9472-d3af22d98715 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:06.052954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 508/1032] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:02:05 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:02:06.130612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 525/1033] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:02:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:02:06.136584 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de24f2-5c56-4991-a010-0b9e8a4a1429 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:06.155075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:06.253973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de24f2-5c56-4991-a010-0b9e8a4a1429 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:06.289281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d150e-16a6-4707-80c9-e11d16bcf339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:06.290074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d150e-16a6-4707-80c9-e11d16bcf339 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:06.290347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d150e-16a6-4707-80c9-e11d16bcf339 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:06.290615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55d150e-16a6-4707-80c9-e11d16bcf339 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:06.302239 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d150e-16a6-4707-80c9-e11d16bcf339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:06.303678 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55d150e-16a6-4707-80c9-e11d16bcf339 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:06.320572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-f55d150e-16a6-4707-80c9-e11d16bcf339 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:02:06.320572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:06.320572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:06.320572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:06.320572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:06.320572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:06.320572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:06.320572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:06.320572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:06.320572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:06.320572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:06.320572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:06.320572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:02:06.320572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:02:06.320572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:06.321529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:06.322445 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:02:06.322445 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:02:06.322445 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:02:06.322445 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:06.322445 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 509/1034] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:02:06.327757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8858aaf-c3e7-4d2d-9472-d3af22d98715 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:06.371165 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:06.431379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8858aaf-c3e7-4d2d-9472-d3af22d98715 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:06.465217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb992e-0cb3-4ef7-a234-0a79a9dc068c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:06.466530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb992e-0cb3-4ef7-a234-0a79a9dc068c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:06.466720 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb992e-0cb3-4ef7-a234-0a79a9dc068c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:06.466980 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb992e-0cb3-4ef7-a234-0a79a9dc068c None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:06.467702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb992e-0cb3-4ef7-a234-0a79a9dc068c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:06.468010 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb992e-0cb3-4ef7-a234-0a79a9dc068c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:06.469405 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbb992e-0cb3-4ef7-a234-0a79a9dc068c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:06.485769 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 526/1035] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:02:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:06.487325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d150e-16a6-4707-80c9-e11d16bcf339 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:06.553765 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:06.594668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55d150e-16a6-4707-80c9-e11d16bcf339 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:06.628124 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa85fa9-60e3-4d30-a96a-ab0ab0a5604c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:06.628946 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa85fa9-60e3-4d30-a96a-ab0ab0a5604c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:06.629231 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa85fa9-60e3-4d30-a96a-ab0ab0a5604c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:06.629507 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa85fa9-60e3-4d30-a96a-ab0ab0a5604c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:06.636360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa85fa9-60e3-4d30-a96a-ab0ab0a5604c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:06.638454 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcb6f3-4435-4673-a951-3d8635d553ac None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:06.638577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa85fa9-60e3-4d30-a96a-ab0ab0a5604c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:06.638767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcb6f3-4435-4673-a951-3d8635d553ac None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:06.639009 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfcb6f3-4435-4673-a951-3d8635d553ac None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:06.640375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 527/1036] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:02:06.645189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e7157-9144-45fb-85ac-153bc3d6572f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:06.645479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e7157-9144-45fb-85ac-153bc3d6572f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:06.645753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e7157-9144-45fb-85ac-153bc3d6572f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:06.655355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-2aa85fa9-60e3-4d30-a96a-ab0ab0a5604c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:02:06.655355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:06.655355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:06.655355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:06.655355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:06.655355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:06.655355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:06.655355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:06.655355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:06.655355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:06.655355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:06.655355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:06.655355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:02:06.655355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:02:06.655355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:06.656264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:06.657691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:02:06.657691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:02:06.657691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:02:06.657691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:02:06.657691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:02:06.657691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:02:06.657691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:02:06.657691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:06.658505 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 510/1037] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:02:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:06.662796 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa85fa9-60e3-4d30-a96a-ab0ab0a5604c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:06.666483 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8a0e7157-9144-45fb-85ac-153bc3d6572f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:06.667734 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8a0e7157-9144-45fb-85ac-153bc3d6572f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:06.737417 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 528/1038] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:02:06.759621 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e7157-9144-45fb-85ac-153bc3d6572f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:06.766504 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa85fa9-60e3-4d30-a96a-ab0ab0a5604c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:06.789928 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:06.794135 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:06.805688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a128ce-1ac2-494a-acab-2a9167065499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:06.806281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a128ce-1ac2-494a-acab-2a9167065499 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:06.806567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a128ce-1ac2-494a-acab-2a9167065499 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:06.806843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a128ce-1ac2-494a-acab-2a9167065499 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:06.807644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a128ce-1ac2-494a-acab-2a9167065499 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:06.807956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a128ce-1ac2-494a-acab-2a9167065499 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:06.809547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a128ce-1ac2-494a-acab-2a9167065499 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:06.821391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 511/1039] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:02:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:06.827776 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a128ce-1ac2-494a-acab-2a9167065499 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:06.867827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e7157-9144-45fb-85ac-153bc3d6572f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:06.902913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b10f8-bb1e-480d-8c0c-52c957efe6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:06.904276 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b10f8-bb1e-480d-8c0c-52c957efe6db None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:06.904568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b10f8-bb1e-480d-8c0c-52c957efe6db None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:06.904826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b10f8-bb1e-480d-8c0c-52c957efe6db None admin] PATH_INFO: `/v3/services/image` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:06.905521 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b10f8-bb1e-480d-8c0c-52c957efe6db None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:06.906898 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197b10f8-bb1e-480d-8c0c-52c957efe6db None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-197b10f8-bb1e-480d-8c0c-52c957efe6db None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:06.918396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:06.919435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:06.920400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:06.920400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:06.920400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:06.920400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:06.920400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:06.920400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:06.920400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:06.920400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:06.920400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:06.920400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:06.920400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:02:06.920400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:02:06.920400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:02:06.920400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:06.920400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 529/1040] 10.222.0.61 () {56 vars in 1105 bytes} [Thu Jul 14 10:02:06 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:02:06.925895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b10f8-bb1e-480d-8c0c-52c957efe6db None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:06.934057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a128ce-1ac2-494a-acab-2a9167065499 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:06.973002 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbbe72-e7b3-4c34-b7ce-a2ad2c98efbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:06.973657 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbbe72-e7b3-4c34-b7ce-a2ad2c98efbd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:06.973936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbbe72-e7b3-4c34-b7ce-a2ad2c98efbd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:06.974209 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bbbe72-e7b3-4c34-b7ce-a2ad2c98efbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:06.975041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbbe72-e7b3-4c34-b7ce-a2ad2c98efbd None admin] RBAC: Adding query filter params (role.id=9cb5d3eed6dc41589bac547cebf73cdb, scope.project.id=57662256880f43018039715a755c087c, user.id=e9f4efa3e2f343c2ac858a277601a7df) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:06.975351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbbe72-e7b3-4c34-b7ce-a2ad2c98efbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:06.976790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bbbe72-e7b3-4c34-b7ce-a2ad2c98efbd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:06.982948 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 512/1041] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:02:06 2022] GET /identity/v3/role_assignments?user.id=e9f4efa3e2f343c2ac858a277601a7df&scope.project.id=57662256880f43018039715a755c087c&role.id=9cb5d3eed6dc41589bac547cebf73cdb => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:07.030452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:07.038785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b10f8-bb1e-480d-8c0c-52c957efe6db None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:07.076572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b32d7-6243-4862-a950-a99c87890b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:07.077194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b32d7-6243-4862-a950-a99c87890b85 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:07.077478 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b32d7-6243-4862-a950-a99c87890b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:07.077748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296b32d7-6243-4862-a950-a99c87890b85 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:07.078462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b32d7-6243-4862-a950-a99c87890b85 None admin] RBAC: Adding query filter params (name=image) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:07.078770 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b32d7-6243-4862-a950-a99c87890b85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:07.080155 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296b32d7-6243-4862-a950-a99c87890b85 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:07.090444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 530/1042] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:07.092720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbbe72-e7b3-4c34-b7ce-a2ad2c98efbd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:07.147589 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:07.201826 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bbbe72-e7b3-4c34-b7ce-a2ad2c98efbd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:07.238567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3401b-e2c2-4742-b05f-5d6f5b4a2409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:07.239264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3401b-e2c2-4742-b05f-5d6f5b4a2409 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:07.239553 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3401b-e2c2-4742-b05f-5d6f5b4a2409 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:07.239821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3401b-e2c2-4742-b05f-5d6f5b4a2409 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:07.240566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3401b-e2c2-4742-b05f-5d6f5b4a2409 None admin] RBAC: Adding query filter params (type=image) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:07.240877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3401b-e2c2-4742-b05f-5d6f5b4a2409 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:07.242318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3401b-e2c2-4742-b05f-5d6f5b4a2409 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:07.251901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 513/1043] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:07 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:07.258037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b32d7-6243-4862-a950-a99c87890b85 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:07.364147 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296b32d7-6243-4862-a950-a99c87890b85 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:07.397863 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4193561d-0e3f-4ed7-9825-0ab3b463ceb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:07.398571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193561d-0e3f-4ed7-9825-0ab3b463ceb3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:07.398853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193561d-0e3f-4ed7-9825-0ab3b463ceb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:07.399121 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4193561d-0e3f-4ed7-9825-0ab3b463ceb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:07.399909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193561d-0e3f-4ed7-9825-0ab3b463ceb3 None admin] RBAC: Adding query filter params (interface=public, service_id=6f774cd73c47461385ac27479b2418ce, region_id=RegionOne) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:07.400214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193561d-0e3f-4ed7-9825-0ab3b463ceb3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:07.401644 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4193561d-0e3f-4ed7-9825-0ab3b463ceb3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:07.411789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 531/1044] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Jul 14 10:02:07 2022] GET /identity/v3/endpoints?service_id=6f774cd73c47461385ac27479b2418ce&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:07.413356 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3401b-e2c2-4742-b05f-5d6f5b4a2409 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:07.517253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3401b-e2c2-4742-b05f-5d6f5b4a2409 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:07.531492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:07.553229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c71f4e2-5958-45e2-b369-81b5201947b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:07.553894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71f4e2-5958-45e2-b369-81b5201947b4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:07.554175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71f4e2-5958-45e2-b369-81b5201947b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:07.554450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c71f4e2-5958-45e2-b369-81b5201947b4 None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:07.555132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71f4e2-5958-45e2-b369-81b5201947b4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:07.556486 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c71f4e2-5958-45e2-b369-81b5201947b4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:07.563204 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 514/1045] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Jul 14 10:02:07 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:02:08.178725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499a678-d6b7-427f-bd9b-19d33f626818 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:08.179243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499a678-d6b7-427f-bd9b-19d33f626818 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:08.179352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6499a678-d6b7-427f-bd9b-19d33f626818 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:08.180761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 532/1046] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:08.186284 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a45056-d8b8-4dd3-8bec-09ac1c23ed00 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:08.186577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a45056-d8b8-4dd3-8bec-09ac1c23ed00 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:08.186856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a45056-d8b8-4dd3-8bec-09ac1c23ed00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:08.203526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-97a45056-d8b8-4dd3-8bec-09ac1c23ed00 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:08.204788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-97a45056-d8b8-4dd3-8bec-09ac1c23ed00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:08.282740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 515/1047] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:08.305890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499a678-d6b7-427f-bd9b-19d33f626818 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:08.331554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:08.408363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6499a678-d6b7-427f-bd9b-19d33f626818 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:08.442341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69b521-7e70-44b2-a5ac-c0f2f53b1af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:08.443054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69b521-7e70-44b2-a5ac-c0f2f53b1af6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:08.443330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69b521-7e70-44b2-a5ac-c0f2f53b1af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:08.443599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b69b521-7e70-44b2-a5ac-c0f2f53b1af6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:08.453914 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b69b521-7e70-44b2-a5ac-c0f2f53b1af6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-0b69b521-7e70-44b2-a5ac-c0f2f53b1af6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:02:08.461492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:08.462437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:08.463321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:08.463321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:08.463321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:08.463321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:08.463321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:02:08.463321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:02:08.463321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:02:08.463321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:02:08.463321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:02:08.463321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:02:08.463321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:08.463321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:08.465257 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 533/1048] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:02:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:08.469069 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a45056-d8b8-4dd3-8bec-09ac1c23ed00 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:08.567880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a45056-d8b8-4dd3-8bec-09ac1c23ed00 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:08.577560 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:08.604721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecace70-99c9-490d-b9f6-8ad29fc54a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:08.605422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecace70-99c9-490d-b9f6-8ad29fc54a31 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:08.605706 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecace70-99c9-490d-b9f6-8ad29fc54a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:08.606543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecace70-99c9-490d-b9f6-8ad29fc54a31 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:08.607315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecace70-99c9-490d-b9f6-8ad29fc54a31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:08.607625 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecace70-99c9-490d-b9f6-8ad29fc54a31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:08.609005 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecace70-99c9-490d-b9f6-8ad29fc54a31 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:08.617599 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 516/1049] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:02:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:08.623086 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69b521-7e70-44b2-a5ac-c0f2f53b1af6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:08.707586 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:08.727580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b69b521-7e70-44b2-a5ac-c0f2f53b1af6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:08.760859 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2366ed3-de0e-48c5-99dc-beafb0ebc84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:08.761759 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2366ed3-de0e-48c5-99dc-beafb0ebc84f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:08.762037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2366ed3-de0e-48c5-99dc-beafb0ebc84f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:08.762404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2366ed3-de0e-48c5-99dc-beafb0ebc84f None admin] PATH_INFO: `/v3/users/cc30c41684b74edb8fc4a2dcb07db696` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:08.775711 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2366ed3-de0e-48c5-99dc-beafb0ebc84f None admin] RBAC: Authorizing `identity:get_user(user_id=cc30c41684b74edb8fc4a2dcb07db696)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:08.777188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2366ed3-de0e-48c5-99dc-beafb0ebc84f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:08.779598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 534/1050] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:02:08 2022] GET /identity/v3/users/cc30c41684b74edb8fc4a2dcb07db696 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:08.785049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecace70-99c9-490d-b9f6-8ad29fc54a31 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:08.883314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecace70-99c9-490d-b9f6-8ad29fc54a31 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:08.917386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e07d24-bc9d-4106-a9e7-6bb41d8c91d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:08.918037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e07d24-bc9d-4106-a9e7-6bb41d8c91d3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:08.918340 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e07d24-bc9d-4106-a9e7-6bb41d8c91d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:08.918619 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e07d24-bc9d-4106-a9e7-6bb41d8c91d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:08.957447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:08.987124 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e07d24-bc9d-4106-a9e7-6bb41d8c91d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:08.988488 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e07d24-bc9d-4106-a9e7-6bb41d8c91d3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:09.025242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffc0e7-4711-45ce-9bd2-afa50c70c9dd None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:09.025535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffc0e7-4711-45ce-9bd2-afa50c70c9dd None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:09.025805 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ffc0e7-4711-45ce-9bd2-afa50c70c9dd None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:09.027299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 535/1051] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:02:09.032806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b08a89-ed63-4568-99ca-41958c3063c6 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:09.033243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b08a89-ed63-4568-99ca-41958c3063c6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:09.033525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b08a89-ed63-4568-99ca-41958c3063c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:09.050778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-21b08a89-ed63-4568-99ca-41958c3063c6 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:09.051976 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-21b08a89-ed63-4568-99ca-41958c3063c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:09.097054 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 517/1052] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:02:08 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:02:09.098355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e07d24-bc9d-4106-a9e7-6bb41d8c91d3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:09.118915 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 536/1053] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:02:09.141703 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b08a89-ed63-4568-99ca-41958c3063c6 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:09.173313 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:09.207761 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e07d24-bc9d-4106-a9e7-6bb41d8c91d3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:09.211208 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:09.250073 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e5b1a-1265-4c11-9453-fba8044d5deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:09.250710 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b08a89-ed63-4568-99ca-41958c3063c6 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:09.250949 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e5b1a-1265-4c11-9453-fba8044d5deb None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:09.251228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e5b1a-1265-4c11-9453-fba8044d5deb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:09.251498 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140e5b1a-1265-4c11-9453-fba8044d5deb None admin] PATH_INFO: `/v3/projects/21b0da207ba64664acf1356c058ee0a9` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:09.252707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e5b1a-1265-4c11-9453-fba8044d5deb None admin] RBAC: Authorizing `identity:get_project(project_id=21b0da207ba64664acf1356c058ee0a9)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:09.254184 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140e5b1a-1265-4c11-9453-fba8044d5deb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:09.260981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 518/1054] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:02:09 2022] GET /identity/v3/projects/21b0da207ba64664acf1356c058ee0a9 => generated 311 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:09.261817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e5b1a-1265-4c11-9453-fba8044d5deb None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:09.288474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0054b-c8b6-431e-9569-161e157db873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:09.289311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0054b-c8b6-431e-9569-161e157db873 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:09.289593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0054b-c8b6-431e-9569-161e157db873 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:09.289866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0054b-c8b6-431e-9569-161e157db873 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:09.300286 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0054b-c8b6-431e-9569-161e157db873 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-05b0054b-c8b6-431e-9569-161e157db873 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:02:09.308408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:09.309907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:09.310811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:09.310811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:09.310811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:09.310811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:09.310811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:02:09.310811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:02:09.310811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:02:09.310811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:02:09.310811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:02:09.310811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:02:09.310811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:02:09.310811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:09.315763 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 537/1055] 10.222.0.61 () {56 vars in 1120 bytes} [Thu Jul 14 10:02:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:02:09.316497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0054b-c8b6-431e-9569-161e157db873 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:09.368742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140e5b1a-1265-4c11-9453-fba8044d5deb None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:09.404859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc9fa5-c74a-40cf-9628-2843a6b67ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:09.405561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc9fa5-c74a-40cf-9628-2843a6b67ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:09.405845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc9fa5-c74a-40cf-9628-2843a6b67ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:09.406123 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc9fa5-c74a-40cf-9628-2843a6b67ed1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:09.406957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc9fa5-c74a-40cf-9628-2843a6b67ed1 None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=21b0da207ba64664acf1356c058ee0a9, user.id=cc30c41684b74edb8fc4a2dcb07db696) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:09.407265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc9fa5-c74a-40cf-9628-2843a6b67ed1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:09.408690 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc9fa5-c74a-40cf-9628-2843a6b67ed1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:09.424960 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 519/1056] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:02:09 2022] GET /identity/v3/role_assignments?user.id=cc30c41684b74edb8fc4a2dcb07db696&scope.project.id=21b0da207ba64664acf1356c058ee0a9&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:09.425203 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0054b-c8b6-431e-9569-161e157db873 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:09.457555 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:09.464202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd69b8-f7ff-4b3d-a7fd-156fa3147187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:09.464896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd69b8-f7ff-4b3d-a7fd-156fa3147187 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:09.465251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd69b8-f7ff-4b3d-a7fd-156fa3147187 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:09.465518 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd69b8-f7ff-4b3d-a7fd-156fa3147187 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:09.466301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd69b8-f7ff-4b3d-a7fd-156fa3147187 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:09.466608 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd69b8-f7ff-4b3d-a7fd-156fa3147187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:09.468026 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd69b8-f7ff-4b3d-a7fd-156fa3147187 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:09.481374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 538/1057] 10.222.0.61 () {56 vars in 1115 bytes} [Thu Jul 14 10:02:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:09.483397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc9fa5-c74a-40cf-9628-2843a6b67ed1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:09.486502 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:09.593186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc9fa5-c74a-40cf-9628-2843a6b67ed1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:09.627222 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a8580-64ac-43f5-8abf-84f359bbc239 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:09.627521 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a8580-64ac-43f5-8abf-84f359bbc239 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:09.627773 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a8580-64ac-43f5-8abf-84f359bbc239 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:09.628379 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171725e-d9b6-40f2-8038-a8cbf4d948ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:09.629300 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 539/1058] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:02:09.629407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171725e-d9b6-40f2-8038-a8cbf4d948ac None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:09.629483 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171725e-d9b6-40f2-8038-a8cbf4d948ac None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:09.629717 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171725e-d9b6-40f2-8038-a8cbf4d948ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:09.633943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4606112-e04d-4387-8cf1-be29a24ab39e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:09.634229 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4606112-e04d-4387-8cf1-be29a24ab39e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:09.634501 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4606112-e04d-4387-8cf1-be29a24ab39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:09.641132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171725e-d9b6-40f2-8038-a8cbf4d948ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:09.642602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171725e-d9b6-40f2-8038-a8cbf4d948ac None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:09.651374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e4606112-e04d-4387-8cf1-be29a24ab39e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:09.652544 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e4606112-e04d-4387-8cf1-be29a24ab39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:09.660491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-a171725e-d9b6-40f2-8038-a8cbf4d948ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:02:09.660491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:09.660491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:09.660491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:09.660491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:09.660491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:09.660491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:09.660491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:09.660491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:09.660491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:09.660491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:09.660491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:09.660491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:02:09.660491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:02:09.660491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:09.661571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:09.662499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:02:09.662499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:02:09.662499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:02:09.662499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:09.662499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 520/1059] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:09.667297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171725e-d9b6-40f2-8038-a8cbf4d948ac None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:09.727211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:09.733070 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 540/1060] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:02:09.744754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4606112-e04d-4387-8cf1-be29a24ab39e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:09.775967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171725e-d9b6-40f2-8038-a8cbf4d948ac None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:09.812394 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c3710-8c36-4d1c-88a9-a72ed84d7300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:09.813091 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c3710-8c36-4d1c-88a9-a72ed84d7300 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:09.813383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c3710-8c36-4d1c-88a9-a72ed84d7300 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:09.813654 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c3710-8c36-4d1c-88a9-a72ed84d7300 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:09.814376 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c3710-8c36-4d1c-88a9-a72ed84d7300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:09.814681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c3710-8c36-4d1c-88a9-a72ed84d7300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:09.816037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c3710-8c36-4d1c-88a9-a72ed84d7300 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:09.828380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 521/1061] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:09.833970 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c3710-8c36-4d1c-88a9-a72ed84d7300 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:09.852252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4606112-e04d-4387-8cf1-be29a24ab39e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:09.879911 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:09.893913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15812fbe-e6b7-4806-bfde-b690be7f7970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:09.894707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15812fbe-e6b7-4806-bfde-b690be7f7970 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:09.894984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15812fbe-e6b7-4806-bfde-b690be7f7970 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:09.895310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15812fbe-e6b7-4806-bfde-b690be7f7970 None admin] PATH_INFO: `/v3/services/image` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:09.895964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15812fbe-e6b7-4806-bfde-b690be7f7970 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:09.897410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15812fbe-e6b7-4806-bfde-b690be7f7970 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-15812fbe-e6b7-4806-bfde-b690be7f7970 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:09.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:09.910982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:09.911829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:09.911829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:09.911829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:09.911829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:09.911829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:09.911829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:09.911829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:09.911829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:09.911829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:09.911829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:09.911829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:02:09.911829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:02:09.911829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 14 10:02:09.911829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:09.911829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 541/1062] 10.222.0.61 () {56 vars in 1105 bytes} [Thu Jul 14 10:02:09 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:02:09.917072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15812fbe-e6b7-4806-bfde-b690be7f7970 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:09.946046 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c3710-8c36-4d1c-88a9-a72ed84d7300 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:09.961825 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:09.983347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753e173-5032-4c4a-95bb-33436ea86f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:09.984139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753e173-5032-4c4a-95bb-33436ea86f57 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:09.984469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753e173-5032-4c4a-95bb-33436ea86f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:09.984699 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753e173-5032-4c4a-95bb-33436ea86f57 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:09.992251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753e173-5032-4c4a-95bb-33436ea86f57 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:09.993817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753e173-5032-4c4a-95bb-33436ea86f57 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-b753e173-5032-4c4a-95bb-33436ea86f57 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:02:10.014515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:10.015509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:10.016467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:10.016467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:10.016467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:02:10.016467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:02:10.016467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:02:10.016467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:02:10.016467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:02:10.016467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:02:10.016467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:02:10.016467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:10.018249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 522/1063] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Jul 14 10:02:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:02:10.023891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753e173-5032-4c4a-95bb-33436ea86f57 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:10.026243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15812fbe-e6b7-4806-bfde-b690be7f7970 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:10.063334 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8fe8a-9d53-452f-a604-9475674ecf53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:10.064004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8fe8a-9d53-452f-a604-9475674ecf53 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:10.064286 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8fe8a-9d53-452f-a604-9475674ecf53 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:10.064559 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f8fe8a-9d53-452f-a604-9475674ecf53 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:10.065303 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8fe8a-9d53-452f-a604-9475674ecf53 None admin] RBAC: Adding query filter params (name=image) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:10.065610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8fe8a-9d53-452f-a604-9475674ecf53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:10.067426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f8fe8a-9d53-452f-a604-9475674ecf53 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:10.077787 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 542/1064] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:10.078747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8fe8a-9d53-452f-a604-9475674ecf53 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:10.129023 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:10.135353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753e173-5032-4c4a-95bb-33436ea86f57 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:10.171374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b046bf0-e1c1-4412-af28-cb25f60b664c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:10.172202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b046bf0-e1c1-4412-af28-cb25f60b664c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:10.172589 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b046bf0-e1c1-4412-af28-cb25f60b664c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:10.172913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b046bf0-e1c1-4412-af28-cb25f60b664c None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:10.174043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b046bf0-e1c1-4412-af28-cb25f60b664c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:10.174352 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b046bf0-e1c1-4412-af28-cb25f60b664c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:10.176024 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b046bf0-e1c1-4412-af28-cb25f60b664c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:10.181132 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f8fe8a-9d53-452f-a604-9475674ecf53 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:10.194017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 523/1065] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Jul 14 10:02:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:10.199405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b046bf0-e1c1-4412-af28-cb25f60b664c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:10.214474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a157a1-a1c4-4a1c-8e1c-17923b58cb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:10.215662 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a157a1-a1c4-4a1c-8e1c-17923b58cb2d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:10.215942 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a157a1-a1c4-4a1c-8e1c-17923b58cb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:10.216212 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a157a1-a1c4-4a1c-8e1c-17923b58cb2d None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:10.216917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a157a1-a1c4-4a1c-8e1c-17923b58cb2d None admin] RBAC: Adding query filter params (type=image) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:10.217246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a157a1-a1c4-4a1c-8e1c-17923b58cb2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:10.218603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a157a1-a1c4-4a1c-8e1c-17923b58cb2d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:10.229666 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 543/1066] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:10.236434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a157a1-a1c4-4a1c-8e1c-17923b58cb2d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:10.241988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:10.323240 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b046bf0-e1c1-4412-af28-cb25f60b664c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:10.355115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a157a1-a1c4-4a1c-8e1c-17923b58cb2d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:10.361449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb48918-197a-4323-ae73-9f2faf0c5819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:10.362253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb48918-197a-4323-ae73-9f2faf0c5819 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:10.362534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb48918-197a-4323-ae73-9f2faf0c5819 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:10.362797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb48918-197a-4323-ae73-9f2faf0c5819 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:10.371031 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:10.374720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb48918-197a-4323-ae73-9f2faf0c5819 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:10.376166 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb48918-197a-4323-ae73-9f2faf0c5819 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:10.393961 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee692782-525a-458a-ac2a-34a30dc17a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:10.394586 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee692782-525a-458a-ac2a-34a30dc17a92 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:10.394693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-aeb48918-197a-4323-ae73-9f2faf0c5819 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:02:10.394693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:10.394693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:10.394693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:10.394693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:10.394693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:10.394693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:10.394693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:10.394693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:10.394693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:10.394693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:10.394693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:10.394693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:02:10.394693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:02:10.394693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:10.395546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:10.396468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:02:10.396468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:02:10.396468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:02:10.396468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:10.396695 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee692782-525a-458a-ac2a-34a30dc17a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:10.396695 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee692782-525a-458a-ac2a-34a30dc17a92 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:10.396695 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee692782-525a-458a-ac2a-34a30dc17a92 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:10.397371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee692782-525a-458a-ac2a-34a30dc17a92 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:10.400478 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 524/1067] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:02:10.401338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb48918-197a-4323-ae73-9f2faf0c5819 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:10.418209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 544/1068] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Jul 14 10:02:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:02:10.507673 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:10.530397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb48918-197a-4323-ae73-9f2faf0c5819 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:10.577790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defca05e-197e-46d2-8467-00294d50375a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:10.578126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defca05e-197e-46d2-8467-00294d50375a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:10.578326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defca05e-197e-46d2-8467-00294d50375a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:10.578600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defca05e-197e-46d2-8467-00294d50375a None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:10.579333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defca05e-197e-46d2-8467-00294d50375a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:10.579643 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defca05e-197e-46d2-8467-00294d50375a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:10.581043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defca05e-197e-46d2-8467-00294d50375a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:10.593334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 525/1069] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:02:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:10.599015 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee692782-525a-458a-ac2a-34a30dc17a92 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:10.705394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee692782-525a-458a-ac2a-34a30dc17a92 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:10.738465 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0f71e-89d1-4440-908b-9e4e971035a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:10.739557 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0f71e-89d1-4440-908b-9e4e971035a2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:10.740042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0f71e-89d1-4440-908b-9e4e971035a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:10.740538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc0f71e-89d1-4440-908b-9e4e971035a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:10.747919 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0f71e-89d1-4440-908b-9e4e971035a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:10.749610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc0f71e-89d1-4440-908b-9e4e971035a2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:10.765428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-abc0f71e-89d1-4440-908b-9e4e971035a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:02:10.765428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:10.765428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:10.765428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:10.765428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:10.765428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:10.765428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:10.765428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:10.765428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:10.765428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:10.765428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:10.765428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:10.765428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:02:10.765428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:02:10.765428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:10.766353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:10.767272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:02:10.767272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:02:10.767272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:02:10.767272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:02:10.767272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:02:10.767272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:02:10.767272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:02:10.767272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:10.769710 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 545/1070] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:02:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:10.775100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defca05e-197e-46d2-8467-00294d50375a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:10.878208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defca05e-197e-46d2-8467-00294d50375a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:10.912107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f626a-f241-4f21-b0b7-38373272c2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:10.912772 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f626a-f241-4f21-b0b7-38373272c2e2 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:10.913087 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f626a-f241-4f21-b0b7-38373272c2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:10.913367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f626a-f241-4f21-b0b7-38373272c2e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:10.914160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f626a-f241-4f21-b0b7-38373272c2e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:10.914472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f626a-f241-4f21-b0b7-38373272c2e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:10.915901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06f626a-f241-4f21-b0b7-38373272c2e2 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:10.925515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:10.933120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 526/1071] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:02:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:10.936397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0f71e-89d1-4440-908b-9e4e971035a2 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:10.968288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:11.044385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc0f71e-89d1-4440-908b-9e4e971035a2 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:11.079214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf20fe-bf90-4780-8666-18d8e0ac539f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:11.080110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf20fe-bf90-4780-8666-18d8e0ac539f None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:11.080438 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf20fe-bf90-4780-8666-18d8e0ac539f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:11.080717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf20fe-bf90-4780-8666-18d8e0ac539f None admin] PATH_INFO: `/v3/projects/57662256880f43018039715a755c087c/users/e9f4efa3e2f343c2ac858a277601a7df/roles/9cb5d3eed6dc41589bac547cebf73cdb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:11.082971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf20fe-bf90-4780-8666-18d8e0ac539f None admin] RBAC: Authorizing `identity:create_grant(project_id=57662256880f43018039715a755c087c, user_id=e9f4efa3e2f343c2ac858a277601a7df, role_id=9cb5d3eed6dc41589bac547cebf73cdb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:11.084408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf20fe-bf90-4780-8666-18d8e0ac539f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:11.104960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 546/1072] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:02:10 2022] PUT /identity/v3/projects/57662256880f43018039715a755c087c/users/e9f4efa3e2f343c2ac858a277601a7df/roles/9cb5d3eed6dc41589bac547cebf73cdb => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:02:11.431475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372ed27-1c1e-46e4-a4a1-00447136ff94 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:11.431805 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372ed27-1c1e-46e4-a4a1-00447136ff94 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:11.432033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372ed27-1c1e-46e4-a4a1-00447136ff94 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:11.434101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 527/1073] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:11.439094 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8079d-27f8-4c0d-9557-9b790abbf129 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:11.439564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8079d-27f8-4c0d-9557-9b790abbf129 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:11.439642 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8079d-27f8-4c0d-9557-9b790abbf129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:11.456662 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ace8079d-27f8-4c0d-9557-9b790abbf129 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:11.457877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ace8079d-27f8-4c0d-9557-9b790abbf129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:11.525500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 547/1074] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:11.546848 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372ed27-1c1e-46e4-a4a1-00447136ff94 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:11.652423 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372ed27-1c1e-46e4-a4a1-00447136ff94 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:11.685860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaad0a1-9f43-4021-9c7d-530837c39ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:11.686639 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaad0a1-9f43-4021-9c7d-530837c39ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:11.686914 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaad0a1-9f43-4021-9c7d-530837c39ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:11.687184 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaad0a1-9f43-4021-9c7d-530837c39ef2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:11.697029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaad0a1-9f43-4021-9c7d-530837c39ef2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-ccaad0a1-9f43-4021-9c7d-530837c39ef2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:02:11.705187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:11.706347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:11.707404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:11.707404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:11.707404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:11.707404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:11.707404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:02:11.707404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:02:11.707404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:02:11.707404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:02:11.707404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:02:11.707404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:02:11.707404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:11.707404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:11.707404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 528/1075] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:02:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:11.712392 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8079d-27f8-4c0d-9557-9b790abbf129 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:11.734654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:11.813228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8079d-27f8-4c0d-9557-9b790abbf129 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:11.847136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6f71c-8555-4685-bf69-984f7bb6d9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:11.847801 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6f71c-8555-4685-bf69-984f7bb6d9ee None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:11.848075 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6f71c-8555-4685-bf69-984f7bb6d9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:11.848341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f6f71c-8555-4685-bf69-984f7bb6d9ee None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:11.849111 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6f71c-8555-4685-bf69-984f7bb6d9ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:11.849412 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6f71c-8555-4685-bf69-984f7bb6d9ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:11.850756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f6f71c-8555-4685-bf69-984f7bb6d9ee None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:11.863437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 548/1076] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:02:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:11.865057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaad0a1-9f43-4021-9c7d-530837c39ef2 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:11.925381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:11.969805 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaad0a1-9f43-4021-9c7d-530837c39ef2 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:12.003303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2c468-34ce-424b-b4cb-21081bae0d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:12.004166 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2c468-34ce-424b-b4cb-21081bae0d2e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:12.004442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2c468-34ce-424b-b4cb-21081bae0d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:12.004712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca2c468-34ce-424b-b4cb-21081bae0d2e None admin] PATH_INFO: `/v3/users/cc30c41684b74edb8fc4a2dcb07db696` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:12.005927 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2c468-34ce-424b-b4cb-21081bae0d2e None admin] RBAC: Authorizing `identity:get_user(user_id=cc30c41684b74edb8fc4a2dcb07db696)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:12.007402 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca2c468-34ce-424b-b4cb-21081bae0d2e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:12.014357 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 529/1077] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:02:11 2022] GET /identity/v3/users/cc30c41684b74edb8fc4a2dcb07db696 => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:12.015615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6f71c-8555-4685-bf69-984f7bb6d9ee None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:12.098210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:12.117185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f6f71c-8555-4685-bf69-984f7bb6d9ee None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:12.150275 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487ad798-eed3-4200-9e16-fa997b386445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:12.151131 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ad798-eed3-4200-9e16-fa997b386445 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:12.151410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ad798-eed3-4200-9e16-fa997b386445 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:12.151681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487ad798-eed3-4200-9e16-fa997b386445 None admin] PATH_INFO: `/v3/projects/21b0da207ba64664acf1356c058ee0a9` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:12.153436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ad798-eed3-4200-9e16-fa997b386445 None admin] RBAC: Authorizing `identity:get_project(project_id=21b0da207ba64664acf1356c058ee0a9)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:12.154850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487ad798-eed3-4200-9e16-fa997b386445 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:12.160960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 549/1078] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:02:12 2022] GET /identity/v3/projects/21b0da207ba64664acf1356c058ee0a9 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:12.162822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2c468-34ce-424b-b4cb-21081bae0d2e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:12.275319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca2c468-34ce-424b-b4cb-21081bae0d2e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:12.302334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:12.311879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd521a99-b377-4bc8-b14b-840d1899da68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:12.312758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd521a99-b377-4bc8-b14b-840d1899da68 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:12.313060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd521a99-b377-4bc8-b14b-840d1899da68 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:12.313325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd521a99-b377-4bc8-b14b-840d1899da68 None admin] PATH_INFO: `/v3/projects/21b0da207ba64664acf1356c058ee0a9/users/cc30c41684b74edb8fc4a2dcb07db696/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:12.315326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd521a99-b377-4bc8-b14b-840d1899da68 None admin] RBAC: Authorizing `identity:create_grant(project_id=21b0da207ba64664acf1356c058ee0a9, user_id=cc30c41684b74edb8fc4a2dcb07db696, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:12.316743 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd521a99-b377-4bc8-b14b-840d1899da68 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:12.336967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 530/1079] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:02:12 2022] PUT /identity/v3/projects/21b0da207ba64664acf1356c058ee0a9/users/cc30c41684b74edb8fc4a2dcb07db696/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:02:12.514848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314213bb-9672-4420-a251-785ab441af49 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:12.515357 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314213bb-9672-4420-a251-785ab441af49 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:12.515604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314213bb-9672-4420-a251-785ab441af49 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:12.516972 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 550/1080] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:12.522181 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914fdd7-ae4d-455b-80d0-180d86032fc1 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:12.522466 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914fdd7-ae4d-455b-80d0-180d86032fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:12.522736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914fdd7-ae4d-455b-80d0-180d86032fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:12.539245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b914fdd7-ae4d-455b-80d0-180d86032fc1 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:12.540489 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b914fdd7-ae4d-455b-80d0-180d86032fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:12.610418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 531/1081] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:02:12.632473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314213bb-9672-4420-a251-785ab441af49 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:12.732912 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314213bb-9672-4420-a251-785ab441af49 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:12.766941 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280292ed-b386-48c9-9ebe-50c793c4391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:12.767502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280292ed-b386-48c9-9ebe-50c793c4391d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:12.767779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280292ed-b386-48c9-9ebe-50c793c4391d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:12.768048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280292ed-b386-48c9-9ebe-50c793c4391d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:12.787727 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:12.837251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280292ed-b386-48c9-9ebe-50c793c4391d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:12.838686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280292ed-b386-48c9-9ebe-50c793c4391d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:12.940037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 551/1082] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:02:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:02:12.941749 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914fdd7-ae4d-455b-80d0-180d86032fc1 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:13.042442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914fdd7-ae4d-455b-80d0-180d86032fc1 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:13.060261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:13.079226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d60f77-da2d-4d93-b05d-3fcc31fe6e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:13.080239 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d60f77-da2d-4d93-b05d-3fcc31fe6e61 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:13.080718 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d60f77-da2d-4d93-b05d-3fcc31fe6e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:13.081175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d60f77-da2d-4d93-b05d-3fcc31fe6e61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:13.092084 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d60f77-da2d-4d93-b05d-3fcc31fe6e61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:13.093688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d60f77-da2d-4d93-b05d-3fcc31fe6e61 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:13.111828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-94d60f77-da2d-4d93-b05d-3fcc31fe6e61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:02:13.111828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:13.111828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:13.111828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:13.111828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:13.111828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:13.111828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:13.111828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:13.111828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:13.111828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:13.111828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:13.111828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:13.111828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:02:13.111828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:02:13.111828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:13.112844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:13.113806 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:02:13.113806 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:02:13.113806 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:02:13.113806 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:13.118881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 532/1083] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:13.120683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280292ed-b386-48c9-9ebe-50c793c4391d None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:13.156511 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b5425-4684-4c66-9195-3effb78af200 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:13.156786 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b5425-4684-4c66-9195-3effb78af200 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:13.157088 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52b5425-4684-4c66-9195-3effb78af200 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:13.158600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 533/1084] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:02:13.163108 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c81bf5-3dbb-44e0-941f-15b2eb261e29 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:13.163391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c81bf5-3dbb-44e0-941f-15b2eb261e29 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:13.163664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c81bf5-3dbb-44e0-941f-15b2eb261e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:13.175955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:13.181200 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-18c81bf5-3dbb-44e0-941f-15b2eb261e29 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:13.182372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-18c81bf5-3dbb-44e0-941f-15b2eb261e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:13.228055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280292ed-b386-48c9-9ebe-50c793c4391d None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:13.263931 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 534/1085] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:02:13.267375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cdd3c-7329-4b93-bb76-ef5793f07225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:13.268270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cdd3c-7329-4b93-bb76-ef5793f07225 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:13.268756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cdd3c-7329-4b93-bb76-ef5793f07225 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:13.269260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7cdd3c-7329-4b93-bb76-ef5793f07225 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:13.270205 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cdd3c-7329-4b93-bb76-ef5793f07225 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:13.270740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cdd3c-7329-4b93-bb76-ef5793f07225 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:13.272303 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7cdd3c-7329-4b93-bb76-ef5793f07225 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:13.293319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c81bf5-3dbb-44e0-941f-15b2eb261e29 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:13.293501 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 552/1086] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:13.390202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:13.393709 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c81bf5-3dbb-44e0-941f-15b2eb261e29 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:13.431368 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c516c-8263-43d4-a435-631a18e0353d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:13.432210 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c516c-8263-43d4-a435-631a18e0353d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:13.432484 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c516c-8263-43d4-a435-631a18e0353d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:13.432749 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653c516c-8263-43d4-a435-631a18e0353d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:13.444775 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653c516c-8263-43d4-a435-631a18e0353d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-653c516c-8263-43d4-a435-631a18e0353d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:02:13.453663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:13.454588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:13.455435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:13.455435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:13.455435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:13.455435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:13.455435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:02:13.455435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:02:13.455435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:02:13.455435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:02:13.455435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:02:13.455435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:02:13.455435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 14 10:02:13.455435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:13.455435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 535/1087] 10.222.0.61 () {56 vars in 1120 bytes} [Thu Jul 14 10:02:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:02:13.460850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cdd3c-7329-4b93-bb76-ef5793f07225 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:13.572899 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7cdd3c-7329-4b93-bb76-ef5793f07225 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:13.584317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:13.607787 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd04f0-64f4-4e30-a46c-4196aa7c1eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:13.608449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd04f0-64f4-4e30-a46c-4196aa7c1eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:13.608785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd04f0-64f4-4e30-a46c-4196aa7c1eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:13.609091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd04f0-64f4-4e30-a46c-4196aa7c1eb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:13.609832 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd04f0-64f4-4e30-a46c-4196aa7c1eb9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:13.610136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd04f0-64f4-4e30-a46c-4196aa7c1eb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:13.611482 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd04f0-64f4-4e30-a46c-4196aa7c1eb9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:13.619109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 553/1088] 10.222.0.61 () {56 vars in 1115 bytes} [Thu Jul 14 10:02:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:13.624722 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c516c-8263-43d4-a435-631a18e0353d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:13.721534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653c516c-8263-43d4-a435-631a18e0353d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:13.753961 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3bb7b-7a1b-4d72-b9ed-3a21848295b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:13.754731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3bb7b-7a1b-4d72-b9ed-3a21848295b7 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:13.755010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3bb7b-7a1b-4d72-b9ed-3a21848295b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:13.755283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f3bb7b-7a1b-4d72-b9ed-3a21848295b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:13.766027 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3bb7b-7a1b-4d72-b9ed-3a21848295b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:13.767448 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f3bb7b-7a1b-4d72-b9ed-3a21848295b7 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:13.783810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-c5f3bb7b-7a1b-4d72-b9ed-3a21848295b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:02:13.783810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:13.783810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:13.783810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:13.783810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:13.783810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:13.783810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:13.783810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:13.783810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:13.783810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:13.783810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:13.783810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:13.783810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:02:13.783810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:02:13.783810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:13.784855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:13.790502 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:02:13.790502 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:02:13.790502 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:02:13.790502 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:13.790502 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 536/1089] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:13.791899 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd04f0-64f4-4e30-a46c-4196aa7c1eb9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:13.888877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd04f0-64f4-4e30-a46c-4196aa7c1eb9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:13.921060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5c0a5-772e-4039-b4a3-4c53c7924d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:13.921717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5c0a5-772e-4039-b4a3-4c53c7924d89 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:13.921997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5c0a5-772e-4039-b4a3-4c53c7924d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:13.922269 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5c0a5-772e-4039-b4a3-4c53c7924d89 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:13.923256 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5c0a5-772e-4039-b4a3-4c53c7924d89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:13.923560 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5c0a5-772e-4039-b4a3-4c53c7924d89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:13.924915 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5c0a5-772e-4039-b4a3-4c53c7924d89 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:13.936850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 554/1090] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:13.942345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3bb7b-7a1b-4d72-b9ed-3a21848295b7 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:13.957849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:14.044184 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f3bb7b-7a1b-4d72-b9ed-3a21848295b7 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:14.077548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46b4be-e10c-4224-94c4-bcb39c5f9ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:14.078334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46b4be-e10c-4224-94c4-bcb39c5f9ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:14.078614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46b4be-e10c-4224-94c4-bcb39c5f9ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:14.078887 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46b4be-e10c-4224-94c4-bcb39c5f9ed5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:14.086220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46b4be-e10c-4224-94c4-bcb39c5f9ed5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:14.087643 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46b4be-e10c-4224-94c4-bcb39c5f9ed5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-fb46b4be-e10c-4224-94c4-bcb39c5f9ed5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:02:14.101967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:14.103059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:14.104449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:14.104449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:14.104449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:02:14.104449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:02:14.104449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:02:14.104449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:02:14.104449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:02:14.104449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:02:14.104449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 14 10:02:14.104449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:14.104449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 537/1091] 10.222.0.61 () {56 vars in 1093 bytes} [Thu Jul 14 10:02:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:14.108859 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5c0a5-772e-4039-b4a3-4c53c7924d89 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:14.139812 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:14.205475 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5c0a5-772e-4039-b4a3-4c53c7924d89 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:14.238115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad379f-0e57-41c2-8747-13895a7a40d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:14.238788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad379f-0e57-41c2-8747-13895a7a40d6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:14.239068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad379f-0e57-41c2-8747-13895a7a40d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:14.239341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad379f-0e57-41c2-8747-13895a7a40d6 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:14.240178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad379f-0e57-41c2-8747-13895a7a40d6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:14.240488 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad379f-0e57-41c2-8747-13895a7a40d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:14.241944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad379f-0e57-41c2-8747-13895a7a40d6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:14.259692 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 555/1092] 10.222.0.61 () {56 vars in 1133 bytes} [Thu Jul 14 10:02:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:14.266165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46b4be-e10c-4224-94c4-bcb39c5f9ed5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:14.336146 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:14.365810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46b4be-e10c-4224-94c4-bcb39c5f9ed5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:14.369312 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946f7a7-d4da-4ce9-982b-72ce58e84a77 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:14.369688 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946f7a7-d4da-4ce9-982b-72ce58e84a77 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:14.370023 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946f7a7-d4da-4ce9-982b-72ce58e84a77 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:14.371440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 556/1093] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:02:14.376193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896b205-4600-4ff0-8606-98a6a5ca143c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:14.376473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896b205-4600-4ff0-8606-98a6a5ca143c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:14.376743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896b205-4600-4ff0-8606-98a6a5ca143c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:14.393586 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e896b205-4600-4ff0-8606-98a6a5ca143c None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:14.394687 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e896b205-4600-4ff0-8606-98a6a5ca143c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:14.399541 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716c98f-ac68-46e1-a35f-8cf058a316dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:14.400188 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716c98f-ac68-46e1-a35f-8cf058a316dc None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:14.400466 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716c98f-ac68-46e1-a35f-8cf058a316dc None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:14.400737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716c98f-ac68-46e1-a35f-8cf058a316dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:14.458994 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 557/1094] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:02:14.469276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716c98f-ac68-46e1-a35f-8cf058a316dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:14.470732 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716c98f-ac68-46e1-a35f-8cf058a316dc None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:14.481483 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896b205-4600-4ff0-8606-98a6a5ca143c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:14.501822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:14.538967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:14.569910 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 538/1095] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:02:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:02:14.576131 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716c98f-ac68-46e1-a35f-8cf058a316dc None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:14.581080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896b205-4600-4ff0-8606-98a6a5ca143c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:14.613763 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d838e0c-01ee-4a07-8639-b69ed2eaf0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:14.614543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d838e0c-01ee-4a07-8639-b69ed2eaf0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:14.614820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d838e0c-01ee-4a07-8639-b69ed2eaf0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:14.615090 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d838e0c-01ee-4a07-8639-b69ed2eaf0b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:14.625272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d838e0c-01ee-4a07-8639-b69ed2eaf0b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-2d838e0c-01ee-4a07-8639-b69ed2eaf0b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:02:14.632717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:14.633701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:14.634563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:14.634563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:14.634563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:14.634563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:14.634563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:02:14.634563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:02:14.634563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:02:14.634563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:02:14.634563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:02:14.634563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:02:14.634563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:14.634563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:14.634563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 558/1096] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:02:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:02:14.639709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d838e0c-01ee-4a07-8639-b69ed2eaf0b2 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:14.673864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716c98f-ac68-46e1-a35f-8cf058a316dc None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:14.706640 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8962b-c757-45d6-b08e-6311cb7bec97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:14.707491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8962b-c757-45d6-b08e-6311cb7bec97 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:14.707772 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8962b-c757-45d6-b08e-6311cb7bec97 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:14.708185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f8962b-c757-45d6-b08e-6311cb7bec97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:14.717680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:14.718568 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8962b-c757-45d6-b08e-6311cb7bec97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:14.720096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f8962b-c757-45d6-b08e-6311cb7bec97 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:14.737377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d838e0c-01ee-4a07-8639-b69ed2eaf0b2 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:14.738420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-81f8962b-c757-45d6-b08e-6311cb7bec97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:02:14.738420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:14.738420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:14.738420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:14.738420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:14.738420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:14.738420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:14.738420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:14.738420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:14.738420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:14.738420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:14.738420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:14.738420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:02:14.738420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:02:14.738420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:14.739358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:14.740290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:02:14.740290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:02:14.740290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:02:14.740290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:14.740290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 539/1097] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:02:14.745190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8962b-c757-45d6-b08e-6311cb7bec97 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:14.769344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:14.771828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21d948-7e7f-4dd8-9712-81f2e0f876d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:14.772531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21d948-7e7f-4dd8-9712-81f2e0f876d6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:14.772813 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21d948-7e7f-4dd8-9712-81f2e0f876d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:14.773210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21d948-7e7f-4dd8-9712-81f2e0f876d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:14.773964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21d948-7e7f-4dd8-9712-81f2e0f876d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:14.774284 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21d948-7e7f-4dd8-9712-81f2e0f876d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:14.775626 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e21d948-7e7f-4dd8-9712-81f2e0f876d6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:14.788438 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 559/1098] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:02:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:14.789614 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21d948-7e7f-4dd8-9712-81f2e0f876d6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:14.844010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f8962b-c757-45d6-b08e-6311cb7bec97 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:14.877321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f39da-500d-4468-86ab-e2954af41419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:14.877979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f39da-500d-4468-86ab-e2954af41419 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:14.878253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f39da-500d-4468-86ab-e2954af41419 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:14.878530 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f39da-500d-4468-86ab-e2954af41419 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:14.879242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f39da-500d-4468-86ab-e2954af41419 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:14.879543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f39da-500d-4468-86ab-e2954af41419 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:14.880893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f39da-500d-4468-86ab-e2954af41419 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:14.885892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21d948-7e7f-4dd8-9712-81f2e0f876d6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:14.892301 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 540/1099] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:02:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:14.897793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f39da-500d-4468-86ab-e2954af41419 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:14.919640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe733a7d-b2a1-4e28-9d0a-3f5f67676926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:14.920508 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe733a7d-b2a1-4e28-9d0a-3f5f67676926 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:14.920789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe733a7d-b2a1-4e28-9d0a-3f5f67676926 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:14.921093 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe733a7d-b2a1-4e28-9d0a-3f5f67676926 None admin] PATH_INFO: `/v3/users/cc30c41684b74edb8fc4a2dcb07db696` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:14.922279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe733a7d-b2a1-4e28-9d0a-3f5f67676926 None admin] RBAC: Authorizing `identity:get_user(user_id=cc30c41684b74edb8fc4a2dcb07db696)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:14.924024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe733a7d-b2a1-4e28-9d0a-3f5f67676926 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:14.925781 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 560/1100] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:02:14 2022] GET /identity/v3/users/cc30c41684b74edb8fc4a2dcb07db696 => generated 285 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:14.931090 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe733a7d-b2a1-4e28-9d0a-3f5f67676926 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:14.942955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:14.980937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:14.998585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f39da-500d-4468-86ab-e2954af41419 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:15.031556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe733a7d-b2a1-4e28-9d0a-3f5f67676926 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:15.032270 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8e003-09c9-43d9-98d4-cd5f1174210f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:15.033089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8e003-09c9-43d9-98d4-cd5f1174210f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:15.033364 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8e003-09c9-43d9-98d4-cd5f1174210f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:15.033634 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b8e003-09c9-43d9-98d4-cd5f1174210f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:15.040275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8e003-09c9-43d9-98d4-cd5f1174210f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:15.041724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b8e003-09c9-43d9-98d4-cd5f1174210f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:15.056114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-43b8e003-09c9-43d9-98d4-cd5f1174210f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:02:15.056114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:15.056114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:15.056114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:15.056114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:15.056114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:15.056114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:15.056114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:15.056114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:15.056114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:15.056114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:15.056114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:15.056114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:02:15.056114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:02:15.056114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:15.057174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:15.058757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:02:15.058757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:02:15.058757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:02:15.058757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:02:15.058757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:02:15.058757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:02:15.058757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:02:15.058757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:15.058757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 541/1101] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:02:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:15.063089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8e003-09c9-43d9-98d4-cd5f1174210f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:15.066199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f78b8-6261-43f1-b5bf-f58793029218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:15.066851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f78b8-6261-43f1-b5bf-f58793029218 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:15.067133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f78b8-6261-43f1-b5bf-f58793029218 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:15.067407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f78b8-6261-43f1-b5bf-f58793029218 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:15.133580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f78b8-6261-43f1-b5bf-f58793029218 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:15.135031 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f78b8-6261-43f1-b5bf-f58793029218 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:15.147232 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:15.161585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b8e003-09c9-43d9-98d4-cd5f1174210f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:15.195436 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d738be4-2df0-4d99-a64e-49716a0f54bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:15.196022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d738be4-2df0-4d99-a64e-49716a0f54bb None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:15.196302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d738be4-2df0-4d99-a64e-49716a0f54bb None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:15.196577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d738be4-2df0-4d99-a64e-49716a0f54bb None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:15.197395 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d738be4-2df0-4d99-a64e-49716a0f54bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:15.197705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d738be4-2df0-4d99-a64e-49716a0f54bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:15.199766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d738be4-2df0-4d99-a64e-49716a0f54bb None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:15.211107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 542/1102] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:02:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:15.217208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d738be4-2df0-4d99-a64e-49716a0f54bb None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:15.220677 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:15.236578 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 561/1103] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:02:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:02:15.242333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f78b8-6261-43f1-b5bf-f58793029218 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:15.323550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d738be4-2df0-4d99-a64e-49716a0f54bb None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:15.340022 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f78b8-6261-43f1-b5bf-f58793029218 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:15.350643 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:15.357056 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f542b-5e59-42fe-9a5a-fff18993be35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:15.357712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f542b-5e59-42fe-9a5a-fff18993be35 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:15.357986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f542b-5e59-42fe-9a5a-fff18993be35 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:15.358254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f542b-5e59-42fe-9a5a-fff18993be35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:15.359081 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f542b-5e59-42fe-9a5a-fff18993be35 None admin] RBAC: Adding query filter params (role.id=9cb5d3eed6dc41589bac547cebf73cdb, scope.project.id=57662256880f43018039715a755c087c, user.id=e9f4efa3e2f343c2ac858a277601a7df) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:15.359390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f542b-5e59-42fe-9a5a-fff18993be35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:15.360807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f542b-5e59-42fe-9a5a-fff18993be35 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:15.366507 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 543/1104] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:02:15 2022] GET /identity/v3/role_assignments?user.id=e9f4efa3e2f343c2ac858a277601a7df&scope.project.id=57662256880f43018039715a755c087c&role.id=9cb5d3eed6dc41589bac547cebf73cdb => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:15.380955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126fc212-9dad-4b61-91b0-f5748b2aac3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:15.381849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fc212-9dad-4b61-91b0-f5748b2aac3a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:15.382125 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fc212-9dad-4b61-91b0-f5748b2aac3a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:15.382394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126fc212-9dad-4b61-91b0-f5748b2aac3a None admin] PATH_INFO: `/v3/projects/21b0da207ba64664acf1356c058ee0a9` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:15.383654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126fc212-9dad-4b61-91b0-f5748b2aac3a None admin] RBAC: Authorizing `identity:get_project(project_id=21b0da207ba64664acf1356c058ee0a9)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:15.385165 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126fc212-9dad-4b61-91b0-f5748b2aac3a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:15.387042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 562/1105] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:02:15 2022] GET /identity/v3/projects/21b0da207ba64664acf1356c058ee0a9 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:15.393376 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f542b-5e59-42fe-9a5a-fff18993be35 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:15.464649 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:15.500326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f542b-5e59-42fe-9a5a-fff18993be35 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:15.534758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad299e9c-263c-4aa6-90de-d4a5a277ab74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:15.535407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad299e9c-263c-4aa6-90de-d4a5a277ab74 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:15.535689 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad299e9c-263c-4aa6-90de-d4a5a277ab74 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:15.535960 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad299e9c-263c-4aa6-90de-d4a5a277ab74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:15.536789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad299e9c-263c-4aa6-90de-d4a5a277ab74 None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=21b0da207ba64664acf1356c058ee0a9, user.id=cc30c41684b74edb8fc4a2dcb07db696) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:15.537135 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad299e9c-263c-4aa6-90de-d4a5a277ab74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:15.538546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad299e9c-263c-4aa6-90de-d4a5a277ab74 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:15.544977 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 544/1106] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:02:15 2022] GET /identity/v3/role_assignments?user.id=cc30c41684b74edb8fc4a2dcb07db696&scope.project.id=21b0da207ba64664acf1356c058ee0a9&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:17.595878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23033422-f422-4c86-ab78-dab68e61060c None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:17.596374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23033422-f422-4c86-ab78-dab68e61060c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:17.596374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23033422-f422-4c86-ab78-dab68e61060c None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:17.597752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 563/1107] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:17.602873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dda9c9-b0cd-4d5d-9c92-e28f29861edb None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:17.603283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dda9c9-b0cd-4d5d-9c92-e28f29861edb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:17.603425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dda9c9-b0cd-4d5d-9c92-e28f29861edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:17.619840 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b2dda9c9-b0cd-4d5d-9c92-e28f29861edb None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:17.621558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b2dda9c9-b0cd-4d5d-9c92-e28f29861edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:17.686469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 545/1108] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:17.711076 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23033422-f422-4c86-ab78-dab68e61060c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:17.815184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23033422-f422-4c86-ab78-dab68e61060c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:17.847996 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbc7eef-54d7-46b4-87b9-954b1771e8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:17.848713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc7eef-54d7-46b4-87b9-954b1771e8f3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:17.849021 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc7eef-54d7-46b4-87b9-954b1771e8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:17.849291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbc7eef-54d7-46b4-87b9-954b1771e8f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:17.850457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc7eef-54d7-46b4-87b9-954b1771e8f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:17.851879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbc7eef-54d7-46b4-87b9-954b1771e8f3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:17.853506 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 564/1109] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:17.859612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dda9c9-b0cd-4d5d-9c92-e28f29861edb None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:17.919315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:17.957262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dda9c9-b0cd-4d5d-9c92-e28f29861edb None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:17.990181 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127443e1-10d2-4cfb-a721-61ecf0f4f8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:17.990955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127443e1-10d2-4cfb-a721-61ecf0f4f8b6 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:17.991239 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127443e1-10d2-4cfb-a721-61ecf0f4f8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:17.991593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127443e1-10d2-4cfb-a721-61ecf0f4f8b6 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:17.992426 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127443e1-10d2-4cfb-a721-61ecf0f4f8b6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:17.993930 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127443e1-10d2-4cfb-a721-61ecf0f4f8b6 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:18.013846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 546/1110] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:02:17 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:02:20.082930 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a400fe6-2ea9-4487-82d4-8a6a2381722e None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:20.083458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a400fe6-2ea9-4487-82d4-8a6a2381722e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:20.083458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a400fe6-2ea9-4487-82d4-8a6a2381722e None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:20.084840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 565/1111] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:20.090601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b16b0-2ef8-4622-8490-32d308f4c601 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:20.091004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b16b0-2ef8-4622-8490-32d308f4c601 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:20.091158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b16b0-2ef8-4622-8490-32d308f4c601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:20.107742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-139b16b0-2ef8-4622-8490-32d308f4c601 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:20.108907 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-139b16b0-2ef8-4622-8490-32d308f4c601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:20.173214 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 547/1112] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:20.195661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a400fe6-2ea9-4487-82d4-8a6a2381722e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:20.242109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:20.295450 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a400fe6-2ea9-4487-82d4-8a6a2381722e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:20.327937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499d09b-f83b-4fc6-a865-7f1e826570a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:20.328820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499d09b-f83b-4fc6-a865-7f1e826570a3 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:20.329199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499d09b-f83b-4fc6-a865-7f1e826570a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:20.329422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0499d09b-f83b-4fc6-a865-7f1e826570a3 None admin] PATH_INFO: `/v3/roles/f4e1d6f34f674f539ea7f2564646c0a5` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:20.330963 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499d09b-f83b-4fc6-a865-7f1e826570a3 None admin] RBAC: Authorizing `identity:get_role(role_id=f4e1d6f34f674f539ea7f2564646c0a5)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:20.332326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0499d09b-f83b-4fc6-a865-7f1e826570a3 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:20.333784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 566/1113] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:02:20 2022] GET /identity/v3/roles/f4e1d6f34f674f539ea7f2564646c0a5 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:20.343905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b16b0-2ef8-4622-8490-32d308f4c601 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:20.391181 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:20.441015 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b16b0-2ef8-4622-8490-32d308f4c601 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:20.474063 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fcc25-4596-4289-8ebc-7e04f76c80fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:20.474913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fcc25-4596-4289-8ebc-7e04f76c80fd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:20.475190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fcc25-4596-4289-8ebc-7e04f76c80fd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:20.475457 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169fcc25-4596-4289-8ebc-7e04f76c80fd None admin] PATH_INFO: `/v3/users/cf0df8c6920045a0840638b89efe9bfa` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:20.489296 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169fcc25-4596-4289-8ebc-7e04f76c80fd None admin] RBAC: Authorizing `identity:get_user(user_id=cf0df8c6920045a0840638b89efe9bfa)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:20.490712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169fcc25-4596-4289-8ebc-7e04f76c80fd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:20.492369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 548/1114] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:02:20 2022] GET /identity/v3/users/cf0df8c6920045a0840638b89efe9bfa => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:20.498077 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499d09b-f83b-4fc6-a865-7f1e826570a3 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:20.592001 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0499d09b-f83b-4fc6-a865-7f1e826570a3 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:20.605563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:20.625439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bae12-d825-42f9-8f75-b4dd0f7ad43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:20.626069 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bae12-d825-42f9-8f75-b4dd0f7ad43a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:20.626346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bae12-d825-42f9-8f75-b4dd0f7ad43a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:20.626615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4bae12-d825-42f9-8f75-b4dd0f7ad43a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:20.690879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bae12-d825-42f9-8f75-b4dd0f7ad43a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:20.692307 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4bae12-d825-42f9-8f75-b4dd0f7ad43a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:20.786694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 567/1115] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:02:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 14 10:02:20.792694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fcc25-4596-4289-8ebc-7e04f76c80fd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:20.888357 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169fcc25-4596-4289-8ebc-7e04f76c80fd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:20.895330 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:20.921176 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7133fb4-8f76-4432-be04-dd8f3cdd30f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:20.922267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7133fb4-8f76-4432-be04-dd8f3cdd30f4 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:20.922544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7133fb4-8f76-4432-be04-dd8f3cdd30f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:20.922815 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7133fb4-8f76-4432-be04-dd8f3cdd30f4 None admin] PATH_INFO: `/v3/projects/21b0da207ba64664acf1356c058ee0a9` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:20.923976 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7133fb4-8f76-4432-be04-dd8f3cdd30f4 None admin] RBAC: Authorizing `identity:get_project(project_id=21b0da207ba64664acf1356c058ee0a9)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:20.925432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7133fb4-8f76-4432-be04-dd8f3cdd30f4 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:20.927101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 549/1116] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:02:20 2022] GET /identity/v3/projects/21b0da207ba64664acf1356c058ee0a9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:20.933576 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bae12-d825-42f9-8f75-b4dd0f7ad43a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:20.947863 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:21.030923 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4bae12-d825-42f9-8f75-b4dd0f7ad43a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:21.063843 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37ed57-862d-4fc9-a120-f161099b6e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:21.064482 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37ed57-862d-4fc9-a120-f161099b6e2e None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:21.064760 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37ed57-862d-4fc9-a120-f161099b6e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:21.065060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37ed57-862d-4fc9-a120-f161099b6e2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:21.065883 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37ed57-862d-4fc9-a120-f161099b6e2e None admin] RBAC: Adding query filter params (role.id=f4e1d6f34f674f539ea7f2564646c0a5, scope.project.id=21b0da207ba64664acf1356c058ee0a9, user.id=cf0df8c6920045a0840638b89efe9bfa) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:21.066187 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37ed57-862d-4fc9-a120-f161099b6e2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:21.067591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf37ed57-862d-4fc9-a120-f161099b6e2e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:21.072824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 568/1117] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:02:20 2022] GET /identity/v3/role_assignments?user.id=cf0df8c6920045a0840638b89efe9bfa&scope.project.id=21b0da207ba64664acf1356c058ee0a9&role.id=f4e1d6f34f674f539ea7f2564646c0a5 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:23.053268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b02af8-c2d0-47e4-83b5-e6db498f92f9 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:23.053796 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b02af8-c2d0-47e4-83b5-e6db498f92f9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:23.053796 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b02af8-c2d0-47e4-83b5-e6db498f92f9 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:23.055184 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 550/1118] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:23.060549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee0c51-7f56-4768-96fe-aa1cfbe6a9e6 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:23.060984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee0c51-7f56-4768-96fe-aa1cfbe6a9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:23.061132 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeee0c51-7f56-4768-96fe-aa1cfbe6a9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:23.077718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-aeee0c51-7f56-4768-96fe-aa1cfbe6a9e6 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:23.078894 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-aeee0c51-7f56-4768-96fe-aa1cfbe6a9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:23.142680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 569/1119] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:23.166031 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b02af8-c2d0-47e4-83b5-e6db498f92f9 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:23.264712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b02af8-c2d0-47e4-83b5-e6db498f92f9 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:23.297824 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeafd3f-e334-41a6-a49f-37cf9abad475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:23.298667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeafd3f-e334-41a6-a49f-37cf9abad475 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:23.298949 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeafd3f-e334-41a6-a49f-37cf9abad475 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:23.299223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeafd3f-e334-41a6-a49f-37cf9abad475 None admin] PATH_INFO: `/v3/roles/f4e1d6f34f674f539ea7f2564646c0a5` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:23.300691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeafd3f-e334-41a6-a49f-37cf9abad475 None admin] RBAC: Authorizing `identity:get_role(role_id=f4e1d6f34f674f539ea7f2564646c0a5)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:23.302086 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeafd3f-e334-41a6-a49f-37cf9abad475 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:23.303493 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 551/1120] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:02:23 2022] GET /identity/v3/roles/f4e1d6f34f674f539ea7f2564646c0a5 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:23.309399 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee0c51-7f56-4768-96fe-aa1cfbe6a9e6 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:23.319208 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:23.404707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeee0c51-7f56-4768-96fe-aa1cfbe6a9e6 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:23.436944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480789d5-4b02-4020-9828-9da21f0a4630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:23.437813 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480789d5-4b02-4020-9828-9da21f0a4630 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:23.438089 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480789d5-4b02-4020-9828-9da21f0a4630 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:23.438357 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480789d5-4b02-4020-9828-9da21f0a4630 None admin] PATH_INFO: `/v3/users/cf0df8c6920045a0840638b89efe9bfa` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:23.439516 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480789d5-4b02-4020-9828-9da21f0a4630 None admin] RBAC: Authorizing `identity:get_user(user_id=cf0df8c6920045a0840638b89efe9bfa)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:23.440948 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480789d5-4b02-4020-9828-9da21f0a4630 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:23.442540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 570/1121] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:02:23 2022] GET /identity/v3/users/cf0df8c6920045a0840638b89efe9bfa => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:23.449183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeafd3f-e334-41a6-a49f-37cf9abad475 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:23.491052 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:23.545248 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeafd3f-e334-41a6-a49f-37cf9abad475 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:23.578593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129dafd-3a79-42d4-89fb-1d43e323ff4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:23.579376 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129dafd-3a79-42d4-89fb-1d43e323ff4e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:23.579650 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129dafd-3a79-42d4-89fb-1d43e323ff4e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:23.579918 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129dafd-3a79-42d4-89fb-1d43e323ff4e None admin] PATH_INFO: `/v3/projects/21b0da207ba64664acf1356c058ee0a9` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:23.581100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129dafd-3a79-42d4-89fb-1d43e323ff4e None admin] RBAC: Authorizing `identity:get_project(project_id=21b0da207ba64664acf1356c058ee0a9)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:23.582503 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4129dafd-3a79-42d4-89fb-1d43e323ff4e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:23.584162 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 552/1122] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:02:23 2022] GET /identity/v3/projects/21b0da207ba64664acf1356c058ee0a9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:23.590122 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480789d5-4b02-4020-9828-9da21f0a4630 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:23.663350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:23.686426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480789d5-4b02-4020-9828-9da21f0a4630 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:23.719255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61f31c1-7549-4d9b-b49b-06fb8c29823e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:23.720128 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f31c1-7549-4d9b-b49b-06fb8c29823e None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:23.720406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f31c1-7549-4d9b-b49b-06fb8c29823e None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:23.720674 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61f31c1-7549-4d9b-b49b-06fb8c29823e None admin] PATH_INFO: `/v3/projects/21b0da207ba64664acf1356c058ee0a9/users/cf0df8c6920045a0840638b89efe9bfa/roles/f4e1d6f34f674f539ea7f2564646c0a5` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:23.722852 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f31c1-7549-4d9b-b49b-06fb8c29823e None admin] RBAC: Authorizing `identity:create_grant(project_id=21b0da207ba64664acf1356c058ee0a9, user_id=cf0df8c6920045a0840638b89efe9bfa, role_id=f4e1d6f34f674f539ea7f2564646c0a5)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:23.724262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61f31c1-7549-4d9b-b49b-06fb8c29823e None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:23.736971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 571/1123] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:02:23 2022] PUT /identity/v3/projects/21b0da207ba64664acf1356c058ee0a9/users/cf0df8c6920045a0840638b89efe9bfa/roles/f4e1d6f34f674f539ea7f2564646c0a5 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:02:25.729314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a88b4-e207-4e40-9e5d-93e250ccf3db None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:25.729859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a88b4-e207-4e40-9e5d-93e250ccf3db None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:25.729859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a88b4-e207-4e40-9e5d-93e250ccf3db None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:25.731213 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 553/1124] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:25.736238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b82ecf-4ca2-4e0a-94c5-08b47880c602 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:25.736676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b82ecf-4ca2-4e0a-94c5-08b47880c602 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:25.736778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b82ecf-4ca2-4e0a-94c5-08b47880c602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:25.753593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-84b82ecf-4ca2-4e0a-94c5-08b47880c602 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:25.754768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-84b82ecf-4ca2-4e0a-94c5-08b47880c602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:25.819938 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 572/1125] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:25.843296 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a88b4-e207-4e40-9e5d-93e250ccf3db None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:25.942255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a88b4-e207-4e40-9e5d-93e250ccf3db None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:25.953283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:25.976092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d1036-067b-4201-9ab9-ddeb8e6302a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:25.976963 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d1036-067b-4201-9ab9-ddeb8e6302a1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:25.977253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d1036-067b-4201-9ab9-ddeb8e6302a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:25.977529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d1036-067b-4201-9ab9-ddeb8e6302a1 None admin] PATH_INFO: `/v3/roles/f4e1d6f34f674f539ea7f2564646c0a5` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:25.979010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d1036-067b-4201-9ab9-ddeb8e6302a1 None admin] RBAC: Authorizing `identity:get_role(role_id=f4e1d6f34f674f539ea7f2564646c0a5)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:25.980410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d1036-067b-4201-9ab9-ddeb8e6302a1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:25.981866 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 554/1126] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:02:25 2022] GET /identity/v3/roles/f4e1d6f34f674f539ea7f2564646c0a5 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:25.987517 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b82ecf-4ca2-4e0a-94c5-08b47880c602 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:26.049104 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:26.083300 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b82ecf-4ca2-4e0a-94c5-08b47880c602 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:26.115803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba24060-51aa-4e3b-ba65-0b470b79a656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:26.116647 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba24060-51aa-4e3b-ba65-0b470b79a656 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:26.116946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba24060-51aa-4e3b-ba65-0b470b79a656 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:26.117218 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba24060-51aa-4e3b-ba65-0b470b79a656 None admin] PATH_INFO: `/v3/users/cf0df8c6920045a0840638b89efe9bfa` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:26.118376 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba24060-51aa-4e3b-ba65-0b470b79a656 None admin] RBAC: Authorizing `identity:get_user(user_id=cf0df8c6920045a0840638b89efe9bfa)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:26.120377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba24060-51aa-4e3b-ba65-0b470b79a656 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:26.121993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 573/1127] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:02:25 2022] GET /identity/v3/users/cf0df8c6920045a0840638b89efe9bfa => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:26.127575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d1036-067b-4201-9ab9-ddeb8e6302a1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:26.226283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d1036-067b-4201-9ab9-ddeb8e6302a1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:26.260219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22069b-1931-4e11-b510-1b5a32c5219c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:26.260863 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22069b-1931-4e11-b510-1b5a32c5219c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:26.261176 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22069b-1931-4e11-b510-1b5a32c5219c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:26.261452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe22069b-1931-4e11-b510-1b5a32c5219c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:26.305604 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:26.327733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22069b-1931-4e11-b510-1b5a32c5219c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:26.329213 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe22069b-1931-4e11-b510-1b5a32c5219c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:26.425016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 555/1128] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:02:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:02:26.430929 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba24060-51aa-4e3b-ba65-0b470b79a656 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:26.526715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba24060-51aa-4e3b-ba65-0b470b79a656 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:26.559383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5065830-7087-45fe-a944-d7c02b6235f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:26.560231 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5065830-7087-45fe-a944-d7c02b6235f4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:26.560513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5065830-7087-45fe-a944-d7c02b6235f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:26.560787 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5065830-7087-45fe-a944-d7c02b6235f4 None admin] PATH_INFO: `/v3/projects/21b0da207ba64664acf1356c058ee0a9` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:26.561991 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5065830-7087-45fe-a944-d7c02b6235f4 None admin] RBAC: Authorizing `identity:get_project(project_id=21b0da207ba64664acf1356c058ee0a9)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:26.563416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5065830-7087-45fe-a944-d7c02b6235f4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:26.565121 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 574/1129] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:02:26 2022] GET /identity/v3/projects/21b0da207ba64664acf1356c058ee0a9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:26.571115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22069b-1931-4e11-b510-1b5a32c5219c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:26.645370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:26.667982 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe22069b-1931-4e11-b510-1b5a32c5219c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:26.700720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b545c89e-445e-4091-b0c2-aa8cf07a6d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:26.701389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545c89e-445e-4091-b0c2-aa8cf07a6d18 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:26.701669 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545c89e-445e-4091-b0c2-aa8cf07a6d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:26.701941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b545c89e-445e-4091-b0c2-aa8cf07a6d18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:26.702764 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545c89e-445e-4091-b0c2-aa8cf07a6d18 None admin] RBAC: Adding query filter params (role.id=f4e1d6f34f674f539ea7f2564646c0a5, scope.project.id=21b0da207ba64664acf1356c058ee0a9, user.id=cf0df8c6920045a0840638b89efe9bfa) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:26.703074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545c89e-445e-4091-b0c2-aa8cf07a6d18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:26.704481 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b545c89e-445e-4091-b0c2-aa8cf07a6d18 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:26.710241 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 556/1130] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:02:26 2022] GET /identity/v3/role_assignments?user.id=cf0df8c6920045a0840638b89efe9bfa&scope.project.id=21b0da207ba64664acf1356c058ee0a9&role.id=f4e1d6f34f674f539ea7f2564646c0a5 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:02:28.759440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d01a64-d981-4b71-aa1c-5847a323f347 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:28.760008 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d01a64-d981-4b71-aa1c-5847a323f347 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:28.760008 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d01a64-d981-4b71-aa1c-5847a323f347 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:28.761372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 575/1131] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:28.766515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74180e-0e34-44de-b450-78de0d5eeb4b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:28.766943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74180e-0e34-44de-b450-78de0d5eeb4b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:28.767050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a74180e-0e34-44de-b450-78de0d5eeb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:28.783877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2a74180e-0e34-44de-b450-78de0d5eeb4b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:28.785091 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2a74180e-0e34-44de-b450-78de0d5eeb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:28.849278 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 557/1132] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:28.872381 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d01a64-d981-4b71-aa1c-5847a323f347 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:28.875278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:28.969754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d01a64-d981-4b71-aa1c-5847a323f347 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:29.002049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b4a60-e6a2-4df7-9b75-f8ebb27accae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:29.002820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b4a60-e6a2-4df7-9b75-f8ebb27accae None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:29.003096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b4a60-e6a2-4df7-9b75-f8ebb27accae None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:29.003366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b4a60-e6a2-4df7-9b75-f8ebb27accae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:29.004521 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b4a60-e6a2-4df7-9b75-f8ebb27accae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:29.005975 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073b4a60-e6a2-4df7-9b75-f8ebb27accae None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:29.007566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 576/1133] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:29.013431 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74180e-0e34-44de-b450-78de0d5eeb4b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:29.080875 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:29.109814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a74180e-0e34-44de-b450-78de0d5eeb4b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:29.142737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a31fb5-a277-46fb-8d23-dd578de3fc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:29.143382 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a31fb5-a277-46fb-8d23-dd578de3fc9f None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:29.143659 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a31fb5-a277-46fb-8d23-dd578de3fc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:29.143931 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a31fb5-a277-46fb-8d23-dd578de3fc9f None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:29.144636 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a31fb5-a277-46fb-8d23-dd578de3fc9f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:29.146080 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a31fb5-a277-46fb-8d23-dd578de3fc9f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:29.171199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 558/1134] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Jul 14 10:02:29 2022] POST /identity/v3/projects => generated 311 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:02:31.188421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57ffd7-b07b-486f-9e78-65ad800cfad4 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:31.188967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57ffd7-b07b-486f-9e78-65ad800cfad4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:31.188967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57ffd7-b07b-486f-9e78-65ad800cfad4 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:31.190343 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 577/1135] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:31.195374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169778d-6a79-4160-9187-15c1917a118e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:31.195789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169778d-6a79-4160-9187-15c1917a118e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:31.196059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169778d-6a79-4160-9187-15c1917a118e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:31.213018 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f169778d-6a79-4160-9187-15c1917a118e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:31.214205 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f169778d-6a79-4160-9187-15c1917a118e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:31.280719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 559/1136] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:31.305318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57ffd7-b07b-486f-9e78-65ad800cfad4 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:31.372047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:31.403069 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57ffd7-b07b-486f-9e78-65ad800cfad4 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:31.435616 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c430c5-f90d-4588-a7a2-f2325685d9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:31.436400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c430c5-f90d-4588-a7a2-f2325685d9bd None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:31.436681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c430c5-f90d-4588-a7a2-f2325685d9bd None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:31.436973 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c430c5-f90d-4588-a7a2-f2325685d9bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:31.438148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c430c5-f90d-4588-a7a2-f2325685d9bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:31.439616 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c430c5-f90d-4588-a7a2-f2325685d9bd None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:31.441186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 578/1137] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:02:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:31.447496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169778d-6a79-4160-9187-15c1917a118e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:31.500580 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:31.544684 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169778d-6a79-4160-9187-15c1917a118e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:31.577922 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c3ff25-b894-4ca1-a82b-c93a19afe2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:31.578581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3ff25-b894-4ca1-a82b-c93a19afe2dd None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:31.578858 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3ff25-b894-4ca1-a82b-c93a19afe2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:31.579155 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c3ff25-b894-4ca1-a82b-c93a19afe2dd None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:31.579866 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3ff25-b894-4ca1-a82b-c93a19afe2dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:31.581300 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c3ff25-b894-4ca1-a82b-c93a19afe2dd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:31.600265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 560/1138] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:02:31 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:02:33.644426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f85703-703d-4ecb-9c75-61be78915f31 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:33.644966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f85703-703d-4ecb-9c75-61be78915f31 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:33.644966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f85703-703d-4ecb-9c75-61be78915f31 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:33.646273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 579/1139] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:33.651535 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870b76e-ad37-448c-8c07-5674691083e3 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:33.651968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870b76e-ad37-448c-8c07-5674691083e3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:33.652095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870b76e-ad37-448c-8c07-5674691083e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:33.669611 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7870b76e-ad37-448c-8c07-5674691083e3 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:33.671049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7870b76e-ad37-448c-8c07-5674691083e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:33.734960 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 561/1140] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:33.758587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f85703-703d-4ecb-9c75-61be78915f31 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:33.855883 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f85703-703d-4ecb-9c75-61be78915f31 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:33.888367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ff7e3-ce96-4741-8a0e-42cff008cbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:33.889165 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ff7e3-ce96-4741-8a0e-42cff008cbbf None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:33.889446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ff7e3-ce96-4741-8a0e-42cff008cbbf None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:33.889720 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5ff7e3-ce96-4741-8a0e-42cff008cbbf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:33.896337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:33.900346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5ff7e3-ce96-4741-8a0e-42cff008cbbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-7b5ff7e3-ce96-4741-8a0e-42cff008cbbf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:02:33.908313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:33.909534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:33.910622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:33.910622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:33.910622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:33.910622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:33.910622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:02:33.910622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:02:33.910622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:02:33.910622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:02:33.910622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:02:33.910622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:02:33.910622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:33.910622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:33.910622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 580/1141] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:02:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:33.915934 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870b76e-ad37-448c-8c07-5674691083e3 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:33.956360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:34.014511 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7870b76e-ad37-448c-8c07-5674691083e3 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:34.048204 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5a2e7-a1b8-49a8-aa6f-9c1673c5ac3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:34.048860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5a2e7-a1b8-49a8-aa6f-9c1673c5ac3a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:34.049176 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5a2e7-a1b8-49a8-aa6f-9c1673c5ac3a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:34.049455 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c5a2e7-a1b8-49a8-aa6f-9c1673c5ac3a None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:34.050200 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c5a2e7-a1b8-49a8-aa6f-9c1673c5ac3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:34.050508 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c5a2e7-a1b8-49a8-aa6f-9c1673c5ac3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:34.051888 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c5a2e7-a1b8-49a8-aa6f-9c1673c5ac3a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:34.059767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 562/1142] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:02:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:34.065401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ff7e3-ce96-4741-8a0e-42cff008cbbf None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:34.160187 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5ff7e3-ce96-4741-8a0e-42cff008cbbf None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:34.192252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d704bb-7fc4-4004-a577-c961167d4fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:34.193132 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d704bb-7fc4-4004-a577-c961167d4fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:34.193412 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d704bb-7fc4-4004-a577-c961167d4fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:34.193682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d704bb-7fc4-4004-a577-c961167d4fa7 None admin] PATH_INFO: `/v3/users/8ab26da79cae4f60b2379a47c736db9d` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:34.206075 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d704bb-7fc4-4004-a577-c961167d4fa7 None admin] RBAC: Authorizing `identity:get_user(user_id=8ab26da79cae4f60b2379a47c736db9d)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:34.207493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d704bb-7fc4-4004-a577-c961167d4fa7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:34.209131 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 581/1143] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:02:34 2022] GET /identity/v3/users/8ab26da79cae4f60b2379a47c736db9d => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:34.215237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5a2e7-a1b8-49a8-aa6f-9c1673c5ac3a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:34.313016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c5a2e7-a1b8-49a8-aa6f-9c1673c5ac3a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:34.314526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:34.347032 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f4544-71e2-4ab8-bf58-b6925776e5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:34.347664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f4544-71e2-4ab8-bf58-b6925776e5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:34.347938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f4544-71e2-4ab8-bf58-b6925776e5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:34.348202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f4544-71e2-4ab8-bf58-b6925776e5c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:34.412183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f4544-71e2-4ab8-bf58-b6925776e5c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:34.413578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f4544-71e2-4ab8-bf58-b6925776e5c1 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:34.506274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 563/1144] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:02:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:02:34.512887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d704bb-7fc4-4004-a577-c961167d4fa7 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:34.568237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:34.612453 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d704bb-7fc4-4004-a577-c961167d4fa7 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:34.645350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45936a5a-6319-4df4-9d50-27733f9aa58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:34.646208 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45936a5a-6319-4df4-9d50-27733f9aa58b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:34.646486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45936a5a-6319-4df4-9d50-27733f9aa58b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:34.646758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45936a5a-6319-4df4-9d50-27733f9aa58b None admin] PATH_INFO: `/v3/projects/8efdd629e3d044dc840b53b43b8e34d6` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:34.647934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45936a5a-6319-4df4-9d50-27733f9aa58b None admin] RBAC: Authorizing `identity:get_project(project_id=8efdd629e3d044dc840b53b43b8e34d6)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:34.649999 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45936a5a-6319-4df4-9d50-27733f9aa58b None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:34.651686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 582/1145] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:02:34 2022] GET /identity/v3/projects/8efdd629e3d044dc840b53b43b8e34d6 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:34.657799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f4544-71e2-4ab8-bf58-b6925776e5c1 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:34.659943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:34.761983 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f4544-71e2-4ab8-bf58-b6925776e5c1 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:34.793866 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14d6932-6574-4aae-9ffd-071eaa02f8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:34.794504 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d6932-6574-4aae-9ffd-071eaa02f8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:34.795038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d6932-6574-4aae-9ffd-071eaa02f8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:34.795312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d6932-6574-4aae-9ffd-071eaa02f8b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:34.796136 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d6932-6574-4aae-9ffd-071eaa02f8b0 None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=8efdd629e3d044dc840b53b43b8e34d6, user.id=8ab26da79cae4f60b2379a47c736db9d) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:34.796444 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d6932-6574-4aae-9ffd-071eaa02f8b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:34.797881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14d6932-6574-4aae-9ffd-071eaa02f8b0 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:34.803166 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 564/1146] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:02:34 2022] GET /identity/v3/role_assignments?user.id=8ab26da79cae4f60b2379a47c736db9d&scope.project.id=8efdd629e3d044dc840b53b43b8e34d6&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:36.789927 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf1e8d-415d-46e8-a76d-12e13b8edfac None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:36.790452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf1e8d-415d-46e8-a76d-12e13b8edfac None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:36.790452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf1e8d-415d-46e8-a76d-12e13b8edfac None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:36.791833 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 583/1147] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:36.797288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfab747-7735-4f34-b8df-d3e333a60715 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:36.797690 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfab747-7735-4f34-b8df-d3e333a60715 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:36.797832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfab747-7735-4f34-b8df-d3e333a60715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:36.814646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3cfab747-7735-4f34-b8df-d3e333a60715 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:36.815798 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3cfab747-7735-4f34-b8df-d3e333a60715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:36.872940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:36.880500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 565/1148] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:36.903745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf1e8d-415d-46e8-a76d-12e13b8edfac None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:37.001640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf1e8d-415d-46e8-a76d-12e13b8edfac None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:37.024556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:37.035233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a75f1-6d1a-44b2-ba69-03c7630aa05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:37.036005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a75f1-6d1a-44b2-ba69-03c7630aa05b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:37.036280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a75f1-6d1a-44b2-ba69-03c7630aa05b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:37.036544 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27a75f1-6d1a-44b2-ba69-03c7630aa05b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:37.047079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27a75f1-6d1a-44b2-ba69-03c7630aa05b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-d27a75f1-6d1a-44b2-ba69-03c7630aa05b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:02:37.054548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:37.055725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:37.056986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:37.056986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:37.056986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:37.056986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:37.056986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:02:37.056986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:02:37.056986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:02:37.056986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:02:37.056986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:02:37.056986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:02:37.056986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:37.056986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:37.056986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 584/1149] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:02:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:37.061723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfab747-7735-4f34-b8df-d3e333a60715 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:37.155644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfab747-7735-4f34-b8df-d3e333a60715 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:37.188135 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3bcb7-acb2-4923-801c-fe62e248eb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:37.188778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3bcb7-acb2-4923-801c-fe62e248eb76 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:37.189083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3bcb7-acb2-4923-801c-fe62e248eb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:37.189355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d3bcb7-acb2-4923-801c-fe62e248eb76 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:37.190091 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3bcb7-acb2-4923-801c-fe62e248eb76 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:37.190392 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3bcb7-acb2-4923-801c-fe62e248eb76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:37.191765 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d3bcb7-acb2-4923-801c-fe62e248eb76 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:37.199574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 566/1150] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:02:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:37.205159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a75f1-6d1a-44b2-ba69-03c7630aa05b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:37.303534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27a75f1-6d1a-44b2-ba69-03c7630aa05b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:37.335558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92213c3c-d0c7-4b2e-8102-0da74faed513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:37.337981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92213c3c-d0c7-4b2e-8102-0da74faed513 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:37.337981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92213c3c-d0c7-4b2e-8102-0da74faed513 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:37.337981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92213c3c-d0c7-4b2e-8102-0da74faed513 None admin] PATH_INFO: `/v3/users/8ab26da79cae4f60b2379a47c736db9d` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:37.338790 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92213c3c-d0c7-4b2e-8102-0da74faed513 None admin] RBAC: Authorizing `identity:get_user(user_id=8ab26da79cae4f60b2379a47c736db9d)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:37.340212 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92213c3c-d0c7-4b2e-8102-0da74faed513 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:37.341872 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 585/1151] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:02:37 2022] GET /identity/v3/users/8ab26da79cae4f60b2379a47c736db9d => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:37.347771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3bcb7-acb2-4923-801c-fe62e248eb76 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:37.404195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:37.444782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d3bcb7-acb2-4923-801c-fe62e248eb76 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:37.477727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52aaca5-6460-4ee2-8501-381ef989c8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:37.478562 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52aaca5-6460-4ee2-8501-381ef989c8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:37.478839 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52aaca5-6460-4ee2-8501-381ef989c8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:37.479105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52aaca5-6460-4ee2-8501-381ef989c8b6 None admin] PATH_INFO: `/v3/projects/8efdd629e3d044dc840b53b43b8e34d6` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:37.480279 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52aaca5-6460-4ee2-8501-381ef989c8b6 None admin] RBAC: Authorizing `identity:get_project(project_id=8efdd629e3d044dc840b53b43b8e34d6)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:37.481721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52aaca5-6460-4ee2-8501-381ef989c8b6 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:37.483401 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 567/1152] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:02:37 2022] GET /identity/v3/projects/8efdd629e3d044dc840b53b43b8e34d6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:37.489237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92213c3c-d0c7-4b2e-8102-0da74faed513 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:37.540707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:37.587181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92213c3c-d0c7-4b2e-8102-0da74faed513 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:37.618885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5c837f-88f6-4aeb-90f1-5aeaf2550396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:37.619762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c837f-88f6-4aeb-90f1-5aeaf2550396 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:37.620041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c837f-88f6-4aeb-90f1-5aeaf2550396 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:37.620310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5c837f-88f6-4aeb-90f1-5aeaf2550396 None admin] PATH_INFO: `/v3/projects/8efdd629e3d044dc840b53b43b8e34d6/users/8ab26da79cae4f60b2379a47c736db9d/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:37.622284 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c837f-88f6-4aeb-90f1-5aeaf2550396 None admin] RBAC: Authorizing `identity:create_grant(project_id=8efdd629e3d044dc840b53b43b8e34d6, user_id=8ab26da79cae4f60b2379a47c736db9d, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:37.623705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5c837f-88f6-4aeb-90f1-5aeaf2550396 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:37.636867 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 586/1153] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:02:37 2022] PUT /identity/v3/projects/8efdd629e3d044dc840b53b43b8e34d6/users/8ab26da79cae4f60b2379a47c736db9d/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:02:39.619451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a412f8-fae9-4926-ac28-56ac7117cbc0 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:39.619944 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a412f8-fae9-4926-ac28-56ac7117cbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:39.619944 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a412f8-fae9-4926-ac28-56ac7117cbc0 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:39.621316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 568/1154] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:39.626355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4304b4-a35c-433c-9878-8ac12faeac15 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:39.626737 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4304b4-a35c-433c-9878-8ac12faeac15 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:39.626915 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4304b4-a35c-433c-9878-8ac12faeac15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:39.643818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1f4304b4-a35c-433c-9878-8ac12faeac15 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:39.645025 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1f4304b4-a35c-433c-9878-8ac12faeac15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:39.709369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 587/1155] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:39.733416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a412f8-fae9-4926-ac28-56ac7117cbc0 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:39.831770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a412f8-fae9-4926-ac28-56ac7117cbc0 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:39.856448 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:39.865358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4702df0-334f-4f5d-a2cf-768525252aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:39.866136 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4702df0-334f-4f5d-a2cf-768525252aea None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:39.866420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4702df0-334f-4f5d-a2cf-768525252aea None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:39.867236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4702df0-334f-4f5d-a2cf-768525252aea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:39.877221 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4702df0-334f-4f5d-a2cf-768525252aea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e4702df0-334f-4f5d-a2cf-768525252aea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:02:39.885305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:39.886415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:39.887470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:39.887470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:39.887470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:39.887470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:39.887470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:02:39.887470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:02:39.887470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:02:39.887470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:02:39.887470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:02:39.887470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:02:39.887470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:39.887470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:39.887470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 569/1156] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:02:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:39.892454 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4304b4-a35c-433c-9878-8ac12faeac15 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:39.933932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:39.988281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4304b4-a35c-433c-9878-8ac12faeac15 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:40.022201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9cc6e-be09-4b6f-a99d-c3379871b826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:40.023365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9cc6e-be09-4b6f-a99d-c3379871b826 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:40.023651 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9cc6e-be09-4b6f-a99d-c3379871b826 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:40.023928 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d9cc6e-be09-4b6f-a99d-c3379871b826 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:40.024677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9cc6e-be09-4b6f-a99d-c3379871b826 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:40.025011 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9cc6e-be09-4b6f-a99d-c3379871b826 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:40.026371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d9cc6e-be09-4b6f-a99d-c3379871b826 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:40.034122 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 588/1157] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:02:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:40.040447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4702df0-334f-4f5d-a2cf-768525252aea None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:40.135907 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4702df0-334f-4f5d-a2cf-768525252aea None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:40.168345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0093958-1d21-4498-a2a4-b221b08f5170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:40.169467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0093958-1d21-4498-a2a4-b221b08f5170 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:40.169747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0093958-1d21-4498-a2a4-b221b08f5170 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:40.170016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0093958-1d21-4498-a2a4-b221b08f5170 None admin] PATH_INFO: `/v3/users/8ab26da79cae4f60b2379a47c736db9d` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:40.171184 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0093958-1d21-4498-a2a4-b221b08f5170 None admin] RBAC: Authorizing `identity:get_user(user_id=8ab26da79cae4f60b2379a47c736db9d)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:40.172625 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0093958-1d21-4498-a2a4-b221b08f5170 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:40.174252 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 570/1158] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:02:40 2022] GET /identity/v3/users/8ab26da79cae4f60b2379a47c736db9d => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:40.179829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9cc6e-be09-4b6f-a99d-c3379871b826 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:40.276543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d9cc6e-be09-4b6f-a99d-c3379871b826 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:40.278826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:40.310614 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebe733-e54b-4900-b101-1d79e5120c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:40.311258 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebe733-e54b-4900-b101-1d79e5120c3a None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:40.311538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebe733-e54b-4900-b101-1d79e5120c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:40.311808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdebe733-e54b-4900-b101-1d79e5120c3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:40.374915 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebe733-e54b-4900-b101-1d79e5120c3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:40.376345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdebe733-e54b-4900-b101-1d79e5120c3a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:40.470585 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 589/1159] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:02:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:02:40.476555 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0093958-1d21-4498-a2a4-b221b08f5170 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:40.530977 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:40.575795 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0093958-1d21-4498-a2a4-b221b08f5170 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:40.608169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268d837-b2b3-4b37-b26a-ea04aadd01db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:40.609041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268d837-b2b3-4b37-b26a-ea04aadd01db None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:40.609324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268d837-b2b3-4b37-b26a-ea04aadd01db None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:40.609597 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268d837-b2b3-4b37-b26a-ea04aadd01db None admin] PATH_INFO: `/v3/projects/8efdd629e3d044dc840b53b43b8e34d6` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:40.610771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268d837-b2b3-4b37-b26a-ea04aadd01db None admin] RBAC: Authorizing `identity:get_project(project_id=8efdd629e3d044dc840b53b43b8e34d6)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:40.612193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2268d837-b2b3-4b37-b26a-ea04aadd01db None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:40.613904 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 571/1160] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:02:40 2022] GET /identity/v3/projects/8efdd629e3d044dc840b53b43b8e34d6 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:40.620094 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebe733-e54b-4900-b101-1d79e5120c3a None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:40.623124 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:40.716302 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdebe733-e54b-4900-b101-1d79e5120c3a None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:40.748600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f48a448-3996-41cf-abd6-cfb629606e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:40.749209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48a448-3996-41cf-abd6-cfb629606e80 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:40.749491 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48a448-3996-41cf-abd6-cfb629606e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:40.749760 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f48a448-3996-41cf-abd6-cfb629606e80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:40.750579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48a448-3996-41cf-abd6-cfb629606e80 None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=8efdd629e3d044dc840b53b43b8e34d6, user.id=8ab26da79cae4f60b2379a47c736db9d) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:40.750886 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48a448-3996-41cf-abd6-cfb629606e80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:40.752293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f48a448-3996-41cf-abd6-cfb629606e80 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:40.758498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 590/1161] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:02:40 2022] GET /identity/v3/role_assignments?user.id=8ab26da79cae4f60b2379a47c736db9d&scope.project.id=8efdd629e3d044dc840b53b43b8e34d6&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:42.763771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5314b10-c788-4404-82a3-8c967932271a None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:42.764294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5314b10-c788-4404-82a3-8c967932271a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:42.764294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5314b10-c788-4404-82a3-8c967932271a None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:42.765721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 572/1162] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:42.770895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e37be-1a43-4bcf-abbd-7cb235ae050f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:42.771311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e37be-1a43-4bcf-abbd-7cb235ae050f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:42.771452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3e37be-1a43-4bcf-abbd-7cb235ae050f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:42.787732 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3d3e37be-1a43-4bcf-abbd-7cb235ae050f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:42.789457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3d3e37be-1a43-4bcf-abbd-7cb235ae050f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:42.849020 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:42.855617 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 591/1163] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:42.878119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5314b10-c788-4404-82a3-8c967932271a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:42.977009 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5314b10-c788-4404-82a3-8c967932271a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:42.998468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:43.010690 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc01c4-840f-4c6e-81b7-72a608fe4974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:43.011330 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc01c4-840f-4c6e-81b7-72a608fe4974 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:43.011610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc01c4-840f-4c6e-81b7-72a608fe4974 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:43.011884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc01c4-840f-4c6e-81b7-72a608fe4974 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:43.077025 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc01c4-840f-4c6e-81b7-72a608fe4974 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:43.078397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bc01c4-840f-4c6e-81b7-72a608fe4974 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:43.172542 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 573/1164] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:02:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:02:43.178736 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e37be-1a43-4bcf-abbd-7cb235ae050f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:43.273200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3e37be-1a43-4bcf-abbd-7cb235ae050f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:43.305937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca149d1-9402-4d6b-816e-86588484a230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:43.306731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca149d1-9402-4d6b-816e-86588484a230 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:43.307012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca149d1-9402-4d6b-816e-86588484a230 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:43.307286 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca149d1-9402-4d6b-816e-86588484a230 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:43.313889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca149d1-9402-4d6b-816e-86588484a230 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:43.315313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca149d1-9402-4d6b-816e-86588484a230 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-7ca149d1-9402-4d6b-816e-86588484a230 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:43.329802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:43.330913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:43.331782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:43.331782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:43.331782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:43.331782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:43.331782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:43.331782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:43.331782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:43.331782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:43.331782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:43.331782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:43.331782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 14 10:02:43.331782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:02:43.331782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 14 10:02:43.331782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:43.331782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 592/1165] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Jul 14 10:02:43 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:43.337417 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc01c4-840f-4c6e-81b7-72a608fe4974 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:43.360133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:43.433536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bc01c4-840f-4c6e-81b7-72a608fe4974 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:43.467279 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5386a88-9748-4803-b986-73536d4a2f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:43.467927 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5386a88-9748-4803-b986-73536d4a2f0a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:43.468205 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5386a88-9748-4803-b986-73536d4a2f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:43.468475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5386a88-9748-4803-b986-73536d4a2f0a None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:43.469219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5386a88-9748-4803-b986-73536d4a2f0a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:43.469523 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5386a88-9748-4803-b986-73536d4a2f0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:43.470874 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5386a88-9748-4803-b986-73536d4a2f0a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:43.477823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 574/1166] 10.222.0.61 () {56 vars in 1115 bytes} [Thu Jul 14 10:02:43 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:45.456765 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ae973-dd8e-4942-aa01-5daa3241ae57 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:45.457340 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ae973-dd8e-4942-aa01-5daa3241ae57 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:45.457340 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837ae973-dd8e-4942-aa01-5daa3241ae57 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:45.458685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 593/1167] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:45.464837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce04cd2-ed22-4515-811a-7f37f17f2ff5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:45.465307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce04cd2-ed22-4515-811a-7f37f17f2ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:45.465415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce04cd2-ed22-4515-811a-7f37f17f2ff5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:45.482275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6ce04cd2-ed22-4515-811a-7f37f17f2ff5 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:45.483449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6ce04cd2-ed22-4515-811a-7f37f17f2ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:45.548051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 575/1168] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:45.571584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ae973-dd8e-4942-aa01-5daa3241ae57 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:45.629047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:45.670257 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837ae973-dd8e-4942-aa01-5daa3241ae57 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:45.702159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a598ec1-892f-417c-a03c-167b49d683b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:45.702799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a598ec1-892f-417c-a03c-167b49d683b6 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:45.703076 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a598ec1-892f-417c-a03c-167b49d683b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:45.703345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a598ec1-892f-417c-a03c-167b49d683b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:45.704080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a598ec1-892f-417c-a03c-167b49d683b6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:45.705453 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a598ec1-892f-417c-a03c-167b49d683b6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:45.719704 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 594/1169] 10.222.0.61 () {60 vars in 1138 bytes} [Thu Jul 14 10:02:45 2022] POST /identity/v3/domains => generated 267 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:02:47.721664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d8515-64d4-4547-97f5-e92f2609adf8 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:47.722183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d8515-64d4-4547-97f5-e92f2609adf8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:47.722296 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d8515-64d4-4547-97f5-e92f2609adf8 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:47.723596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 576/1170] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:47.729537 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c99af5-360a-41c3-bbe9-b7f9eb04b3af None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:47.729970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c99af5-360a-41c3-bbe9-b7f9eb04b3af None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:47.730104 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c99af5-360a-41c3-bbe9-b7f9eb04b3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:47.746990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d3c99af5-360a-41c3-bbe9-b7f9eb04b3af None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:47.748310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d3c99af5-360a-41c3-bbe9-b7f9eb04b3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:47.813520 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 595/1171] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:47.838259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d8515-64d4-4547-97f5-e92f2609adf8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:47.855731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:47.935641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d8515-64d4-4547-97f5-e92f2609adf8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:47.969168 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ead2e5-5082-4edd-ac66-b77103125c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:47.970006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ead2e5-5082-4edd-ac66-b77103125c61 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:47.970284 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ead2e5-5082-4edd-ac66-b77103125c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:47.970552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ead2e5-5082-4edd-ac66-b77103125c61 None admin] PATH_INFO: `/v3/domains/0c007503e4674c1999bf683ab10c0999` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:47.980602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ead2e5-5082-4edd-ac66-b77103125c61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0c007503e4674c1999bf683ab10c0999)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:47.982009 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ead2e5-5082-4edd-ac66-b77103125c61 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:47.983617 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 577/1172] 10.222.0.61 () {56 vars in 1183 bytes} [Thu Jul 14 10:02:47 2022] GET /identity/v3/domains/0c007503e4674c1999bf683ab10c0999 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:47.989425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c99af5-360a-41c3-bbe9-b7f9eb04b3af None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:48.037006 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:48.086289 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c99af5-360a-41c3-bbe9-b7f9eb04b3af None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:48.118185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83756aa-b554-4597-b20c-899c5017d00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:48.118837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83756aa-b554-4597-b20c-899c5017d00a None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:48.119117 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83756aa-b554-4597-b20c-899c5017d00a None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:48.119391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83756aa-b554-4597-b20c-899c5017d00a None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:48.120092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83756aa-b554-4597-b20c-899c5017d00a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:48.121548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83756aa-b554-4597-b20c-899c5017d00a None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:48.145879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 596/1173] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Jul 14 10:02:47 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:02:50.198756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d652b0-3af9-49c0-ae9b-ad762cc7808b None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:50.199284 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d652b0-3af9-49c0-ae9b-ad762cc7808b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:50.199284 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d652b0-3af9-49c0-ae9b-ad762cc7808b None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:50.200645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 578/1174] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:50.205880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a5372-56d3-4644-b17a-b6f54daf6647 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:50.206281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a5372-56d3-4644-b17a-b6f54daf6647 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:50.206435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a5372-56d3-4644-b17a-b6f54daf6647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:50.223233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4c1a5372-56d3-4644-b17a-b6f54daf6647 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:50.224402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4c1a5372-56d3-4644-b17a-b6f54daf6647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:50.289019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 597/1175] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:50.312911 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d652b0-3af9-49c0-ae9b-ad762cc7808b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:50.390901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:50.414983 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d652b0-3af9-49c0-ae9b-ad762cc7808b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:50.448499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b180c58-90bb-4ed7-82c3-fb32164949ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:50.449430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b180c58-90bb-4ed7-82c3-fb32164949ef None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:50.449666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b180c58-90bb-4ed7-82c3-fb32164949ef None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:50.449940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b180c58-90bb-4ed7-82c3-fb32164949ef None admin] PATH_INFO: `/v3/domains/0c007503e4674c1999bf683ab10c0999` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:50.451114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b180c58-90bb-4ed7-82c3-fb32164949ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=0c007503e4674c1999bf683ab10c0999)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:50.452546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b180c58-90bb-4ed7-82c3-fb32164949ef None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:50.454180 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 579/1176] 10.222.0.61 () {56 vars in 1183 bytes} [Thu Jul 14 10:02:50 2022] GET /identity/v3/domains/0c007503e4674c1999bf683ab10c0999 => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:50.459811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a5372-56d3-4644-b17a-b6f54daf6647 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:50.487877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:50.557028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a5372-56d3-4644-b17a-b6f54daf6647 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:50.589493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173e9dcb-64a7-4029-87d1-97c12a4873da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:50.590161 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e9dcb-64a7-4029-87d1-97c12a4873da None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:50.590442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e9dcb-64a7-4029-87d1-97c12a4873da None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:50.590714 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173e9dcb-64a7-4029-87d1-97c12a4873da None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:50.591431 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e9dcb-64a7-4029-87d1-97c12a4873da None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:50.592847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173e9dcb-64a7-4029-87d1-97c12a4873da None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:50.611694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 598/1177] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:02:50 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:02:52.626791 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d8a8d-c46c-4c2c-b888-a3d61dcd539b None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:52.627300 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d8a8d-c46c-4c2c-b888-a3d61dcd539b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:52.627300 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d8a8d-c46c-4c2c-b888-a3d61dcd539b None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:52.628620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 580/1178] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:52.633675 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fc046-0cb6-4f9c-9cf2-e8611053f989 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:52.634076 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fc046-0cb6-4f9c-9cf2-e8611053f989 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:52.634233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fc046-0cb6-4f9c-9cf2-e8611053f989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:52.650579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-326fc046-0cb6-4f9c-9cf2-e8611053f989 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:52.651758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-326fc046-0cb6-4f9c-9cf2-e8611053f989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:52.717697 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 599/1179] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:52.740770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d8a8d-c46c-4c2c-b888-a3d61dcd539b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:52.839239 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d8a8d-c46c-4c2c-b888-a3d61dcd539b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:52.871734 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b463448-e75e-4237-bc79-3e6975a2d4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:52.872508 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b463448-e75e-4237-bc79-3e6975a2d4cd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:52.872788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b463448-e75e-4237-bc79-3e6975a2d4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:52.873094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b463448-e75e-4237-bc79-3e6975a2d4cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:52.882915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b463448-e75e-4237-bc79-3e6975a2d4cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-8b463448-e75e-4237-bc79-3e6975a2d4cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:02:52.890385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:52.891514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:52.892414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:52.892414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:52.892414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:52.892414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:52.892414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:02:52.892414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:02:52.892414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:02:52.892414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:02:52.892414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:02:52.892414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:02:52.892414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:52.892414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:52.892414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 581/1180] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:02:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:52.897853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fc046-0cb6-4f9c-9cf2-e8611053f989 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:52.915143 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:52.995288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fc046-0cb6-4f9c-9cf2-e8611053f989 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:53.028059 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75304460-10b9-4068-b565-242a1e5d2793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:53.028717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75304460-10b9-4068-b565-242a1e5d2793 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:53.029025 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75304460-10b9-4068-b565-242a1e5d2793 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:53.029301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75304460-10b9-4068-b565-242a1e5d2793 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:53.030047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75304460-10b9-4068-b565-242a1e5d2793 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:53.030355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75304460-10b9-4068-b565-242a1e5d2793 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:53.031717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75304460-10b9-4068-b565-242a1e5d2793 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:53.040044 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 600/1181] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:02:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:53.045996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b463448-e75e-4237-bc79-3e6975a2d4cd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:53.049707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:53.144078 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b463448-e75e-4237-bc79-3e6975a2d4cd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:53.176695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fdb19-d277-444c-9aa9-e13b3e2b84a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:53.177609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fdb19-d277-444c-9aa9-e13b3e2b84a6 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:53.177893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fdb19-d277-444c-9aa9-e13b3e2b84a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:53.178167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fdb19-d277-444c-9aa9-e13b3e2b84a6 None admin] PATH_INFO: `/v3/users/a7380dfa594c4eb1a174c0a259d9f17b` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:53.190787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fdb19-d277-444c-9aa9-e13b3e2b84a6 None admin] RBAC: Authorizing `identity:get_user(user_id=a7380dfa594c4eb1a174c0a259d9f17b)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:53.192234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fdb19-d277-444c-9aa9-e13b3e2b84a6 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:53.194439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 582/1182] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:02:53 2022] GET /identity/v3/users/a7380dfa594c4eb1a174c0a259d9f17b => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:53.200108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75304460-10b9-4068-b565-242a1e5d2793 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:53.282960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:53.303999 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75304460-10b9-4068-b565-242a1e5d2793 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:53.336547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042321b5-5b66-4ad4-b79c-81b1beadf811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:53.337208 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042321b5-5b66-4ad4-b79c-81b1beadf811 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:53.337488 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042321b5-5b66-4ad4-b79c-81b1beadf811 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:53.337759 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042321b5-5b66-4ad4-b79c-81b1beadf811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:53.400740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042321b5-5b66-4ad4-b79c-81b1beadf811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:53.402188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042321b5-5b66-4ad4-b79c-81b1beadf811 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:53.473486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:53.494883 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 601/1183] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:02:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:02:53.500736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fdb19-d277-444c-9aa9-e13b3e2b84a6 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:53.561157 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:53.600570 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fdb19-d277-444c-9aa9-e13b3e2b84a6 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:53.633589 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c88eb-2a2b-4a59-98fe-fca9c70780f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:53.634434 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c88eb-2a2b-4a59-98fe-fca9c70780f3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:53.634712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c88eb-2a2b-4a59-98fe-fca9c70780f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:53.634982 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c88eb-2a2b-4a59-98fe-fca9c70780f3 None admin] PATH_INFO: `/v3/projects/ce5264b4ccd24026a2cdefadb107d54c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:53.636150 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c88eb-2a2b-4a59-98fe-fca9c70780f3 None admin] RBAC: Authorizing `identity:get_project(project_id=ce5264b4ccd24026a2cdefadb107d54c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:53.637608 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c88eb-2a2b-4a59-98fe-fca9c70780f3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:53.639288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 583/1184] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:02:53 2022] GET /identity/v3/projects/ce5264b4ccd24026a2cdefadb107d54c => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:53.645317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042321b5-5b66-4ad4-b79c-81b1beadf811 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:53.740979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042321b5-5b66-4ad4-b79c-81b1beadf811 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:53.773857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aba2aa-6ad0-484a-a6ba-69eaa7d004f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:53.774329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aba2aa-6ad0-484a-a6ba-69eaa7d004f7 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:53.774611 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aba2aa-6ad0-484a-a6ba-69eaa7d004f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:53.774883 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aba2aa-6ad0-484a-a6ba-69eaa7d004f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:53.775711 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aba2aa-6ad0-484a-a6ba-69eaa7d004f7 None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=ce5264b4ccd24026a2cdefadb107d54c, user.id=a7380dfa594c4eb1a174c0a259d9f17b) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:53.776025 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aba2aa-6ad0-484a-a6ba-69eaa7d004f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:53.777472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aba2aa-6ad0-484a-a6ba-69eaa7d004f7 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:53.782792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 602/1185] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:02:53 2022] GET /identity/v3/role_assignments?user.id=a7380dfa594c4eb1a174c0a259d9f17b&scope.project.id=ce5264b4ccd24026a2cdefadb107d54c&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:55.758582 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c8f72-c50a-46aa-a996-e820be4c804a None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:55.759156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c8f72-c50a-46aa-a996-e820be4c804a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:55.759156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637c8f72-c50a-46aa-a996-e820be4c804a None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:55.760507 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 584/1186] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:55.765629 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dbaad-e51b-4494-b9be-b99fa020aa81 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:55.766054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dbaad-e51b-4494-b9be-b99fa020aa81 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:55.766186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960dbaad-e51b-4494-b9be-b99fa020aa81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:55.782979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-960dbaad-e51b-4494-b9be-b99fa020aa81 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:55.784146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-960dbaad-e51b-4494-b9be-b99fa020aa81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:55.819022 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:55.849728 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 603/1187] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:55.872396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c8f72-c50a-46aa-a996-e820be4c804a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:55.971618 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637c8f72-c50a-46aa-a996-e820be4c804a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:55.999840 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:56.005940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08487090-ea56-4df9-9b25-acdb1eb30a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:56.006721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08487090-ea56-4df9-9b25-acdb1eb30a28 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:56.007002 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08487090-ea56-4df9-9b25-acdb1eb30a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:56.007276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08487090-ea56-4df9-9b25-acdb1eb30a28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:56.017283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08487090-ea56-4df9-9b25-acdb1eb30a28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-08487090-ea56-4df9-9b25-acdb1eb30a28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:02:56.025305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:56.026538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:56.027666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:56.027666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:56.027666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:56.027666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:56.027666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:02:56.027666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:02:56.027666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:02:56.027666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:02:56.027666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:02:56.027666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:02:56.027666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:56.027666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:02:56.027666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 585/1188] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:02:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:56.032539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dbaad-e51b-4494-b9be-b99fa020aa81 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:56.125742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960dbaad-e51b-4494-b9be-b99fa020aa81 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:56.158202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f2236-4608-40c9-be75-2ef28ffe5c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:56.158847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f2236-4608-40c9-be75-2ef28ffe5c72 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:56.159122 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f2236-4608-40c9-be75-2ef28ffe5c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:56.159389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f2236-4608-40c9-be75-2ef28ffe5c72 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:56.160120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f2236-4608-40c9-be75-2ef28ffe5c72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:56.160420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f2236-4608-40c9-be75-2ef28ffe5c72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:56.161816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f2236-4608-40c9-be75-2ef28ffe5c72 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:56.169757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 604/1189] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:02:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:56.175317 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08487090-ea56-4df9-9b25-acdb1eb30a28 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:56.273013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08487090-ea56-4df9-9b25-acdb1eb30a28 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:56.304938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c454492-4b7b-466a-a9ef-d9297d82407e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:56.305820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c454492-4b7b-466a-a9ef-d9297d82407e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:56.306100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c454492-4b7b-466a-a9ef-d9297d82407e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:56.306372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c454492-4b7b-466a-a9ef-d9297d82407e None admin] PATH_INFO: `/v3/users/a7380dfa594c4eb1a174c0a259d9f17b` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:56.307531 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c454492-4b7b-466a-a9ef-d9297d82407e None admin] RBAC: Authorizing `identity:get_user(user_id=a7380dfa594c4eb1a174c0a259d9f17b)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:56.308977 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c454492-4b7b-466a-a9ef-d9297d82407e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:56.310592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 586/1190] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:02:56 2022] GET /identity/v3/users/a7380dfa594c4eb1a174c0a259d9f17b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:56.316472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f2236-4608-40c9-be75-2ef28ffe5c72 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:56.350428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:56.414839 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f2236-4608-40c9-be75-2ef28ffe5c72 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:56.447658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8824f8-3b62-409d-9ba7-83ba88a1cbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:56.448505 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8824f8-3b62-409d-9ba7-83ba88a1cbf4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:56.448788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8824f8-3b62-409d-9ba7-83ba88a1cbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:56.449093 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8824f8-3b62-409d-9ba7-83ba88a1cbf4 None admin] PATH_INFO: `/v3/projects/ce5264b4ccd24026a2cdefadb107d54c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:56.450266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8824f8-3b62-409d-9ba7-83ba88a1cbf4 None admin] RBAC: Authorizing `identity:get_project(project_id=ce5264b4ccd24026a2cdefadb107d54c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:56.451686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8824f8-3b62-409d-9ba7-83ba88a1cbf4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:56.453411 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 605/1191] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:02:56 2022] GET /identity/v3/projects/ce5264b4ccd24026a2cdefadb107d54c => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:56.459474 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c454492-4b7b-466a-a9ef-d9297d82407e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:56.517339 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:56.558884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c454492-4b7b-466a-a9ef-d9297d82407e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:56.590657 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562fef3d-e8a0-46aa-97a4-c7fd46672166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:56.591525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fef3d-e8a0-46aa-97a4-c7fd46672166 None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:56.591803 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fef3d-e8a0-46aa-97a4-c7fd46672166 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:56.592075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562fef3d-e8a0-46aa-97a4-c7fd46672166 None admin] PATH_INFO: `/v3/projects/ce5264b4ccd24026a2cdefadb107d54c/users/a7380dfa594c4eb1a174c0a259d9f17b/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:56.594003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fef3d-e8a0-46aa-97a4-c7fd46672166 None admin] RBAC: Authorizing `identity:create_grant(project_id=ce5264b4ccd24026a2cdefadb107d54c, user_id=a7380dfa594c4eb1a174c0a259d9f17b, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:56.595413 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562fef3d-e8a0-46aa-97a4-c7fd46672166 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:56.608220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 587/1192] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:02:56 2022] PUT /identity/v3/projects/ce5264b4ccd24026a2cdefadb107d54c/users/a7380dfa594c4eb1a174c0a259d9f17b/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:02:58.611674 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51c0e0-4707-4bd0-bfcd-1c316a5daf80 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:58.612181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51c0e0-4707-4bd0-bfcd-1c316a5daf80 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:58.612181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51c0e0-4707-4bd0-bfcd-1c316a5daf80 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:58.613644 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 606/1193] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:02:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:02:58.619286 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d405793-469d-4b78-9924-e2eb6bd4e761 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:58.619714 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d405793-469d-4b78-9924-e2eb6bd4e761 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:58.619846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d405793-469d-4b78-9924-e2eb6bd4e761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:58.636110 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-5d405793-469d-4b78-9924-e2eb6bd4e761 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:02:58.637324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5d405793-469d-4b78-9924-e2eb6bd4e761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:02:58.702525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 588/1194] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:02:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:02:58.726367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51c0e0-4707-4bd0-bfcd-1c316a5daf80 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:58.822180 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba51c0e0-4707-4bd0-bfcd-1c316a5daf80 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:58.823632 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:58.855790 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca1b67-a54d-40fc-9d42-1663836ae849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:58.856563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca1b67-a54d-40fc-9d42-1663836ae849 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:58.856841 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca1b67-a54d-40fc-9d42-1663836ae849 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:58.857141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca1b67-a54d-40fc-9d42-1663836ae849 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:58.866925 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca1b67-a54d-40fc-9d42-1663836ae849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-9eca1b67-a54d-40fc-9d42-1663836ae849 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:02:58.874528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:02:58.875628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:02:58.876619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:02:58.876619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:02:58.876619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:02:58.876619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:02:58.876619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:02:58.876619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:02:58.876619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:02:58.876619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:02:58.876619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:02:58.876619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:02:58.876619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:02:58.876619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:02:58.876619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 607/1195] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:02:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:02:58.881628 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d405793-469d-4b78-9924-e2eb6bd4e761 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:58.929478 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:58.980015 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d405793-469d-4b78-9924-e2eb6bd4e761 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:59.016745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc4361-dbae-4b67-ba18-51935ad0cf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:59.017443 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc4361-dbae-4b67-ba18-51935ad0cf7e None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:59.017725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc4361-dbae-4b67-ba18-51935ad0cf7e None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:59.018001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc4361-dbae-4b67-ba18-51935ad0cf7e None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:59.018742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc4361-dbae-4b67-ba18-51935ad0cf7e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:59.019048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc4361-dbae-4b67-ba18-51935ad0cf7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:59.020955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc4361-dbae-4b67-ba18-51935ad0cf7e None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:59.028824 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 589/1196] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:02:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:59.034551 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca1b67-a54d-40fc-9d42-1663836ae849 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:59.129604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca1b67-a54d-40fc-9d42-1663836ae849 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:59.161838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008e26f-bec4-4fb6-9acb-0592d0d07221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:59.162680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008e26f-bec4-4fb6-9acb-0592d0d07221 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:59.162956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008e26f-bec4-4fb6-9acb-0592d0d07221 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:59.163224 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7008e26f-bec4-4fb6-9acb-0592d0d07221 None admin] PATH_INFO: `/v3/users/a7380dfa594c4eb1a174c0a259d9f17b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:59.164385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008e26f-bec4-4fb6-9acb-0592d0d07221 None admin] RBAC: Authorizing `identity:get_user(user_id=a7380dfa594c4eb1a174c0a259d9f17b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:59.165818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7008e26f-bec4-4fb6-9acb-0592d0d07221 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:59.167646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 608/1197] 10.222.0.61 () {56 vars in 1177 bytes} [Thu Jul 14 10:02:59 2022] GET /identity/v3/users/a7380dfa594c4eb1a174c0a259d9f17b => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:59.173145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc4361-dbae-4b67-ba18-51935ad0cf7e None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:59.268513 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc4361-dbae-4b67-ba18-51935ad0cf7e None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:59.278377 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:59.302285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5e18f-b541-4016-a4da-a097db6a1018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:59.302928 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5e18f-b541-4016-a4da-a097db6a1018 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:59.303208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5e18f-b541-4016-a4da-a097db6a1018 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:59.303479 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b5e18f-b541-4016-a4da-a097db6a1018 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:59.367391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5e18f-b541-4016-a4da-a097db6a1018 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:59.368816 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b5e18f-b541-4016-a4da-a097db6a1018 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:59.463592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 590/1198] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:02:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:02:59.469422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008e26f-bec4-4fb6-9acb-0592d0d07221 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:59.499136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:59.565509 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7008e26f-bec4-4fb6-9acb-0592d0d07221 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:59.597153 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43591ba-0ac7-4dc9-8c8d-982f7db00191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:59.597987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43591ba-0ac7-4dc9-8c8d-982f7db00191 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:59.598263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43591ba-0ac7-4dc9-8c8d-982f7db00191 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:59.598577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43591ba-0ac7-4dc9-8c8d-982f7db00191 None admin] PATH_INFO: `/v3/projects/ce5264b4ccd24026a2cdefadb107d54c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:59.600229 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43591ba-0ac7-4dc9-8c8d-982f7db00191 None admin] RBAC: Authorizing `identity:get_project(project_id=ce5264b4ccd24026a2cdefadb107d54c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:59.601676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43591ba-0ac7-4dc9-8c8d-982f7db00191 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:59.603342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 609/1199] 10.222.0.61 () {56 vars in 1186 bytes} [Thu Jul 14 10:02:59 2022] GET /identity/v3/projects/ce5264b4ccd24026a2cdefadb107d54c => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:02:59.609341 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5e18f-b541-4016-a4da-a097db6a1018 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:02:59.620787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:02:59.705111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b5e18f-b541-4016-a4da-a097db6a1018 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:02:59.738138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c023c01-21f0-4eba-a703-8520c3ca44c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:02:59.738720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c023c01-21f0-4eba-a703-8520c3ca44c8 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:02:59.738999 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c023c01-21f0-4eba-a703-8520c3ca44c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:02:59.739299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c023c01-21f0-4eba-a703-8520c3ca44c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:02:59.740115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c023c01-21f0-4eba-a703-8520c3ca44c8 None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=ce5264b4ccd24026a2cdefadb107d54c, user.id=a7380dfa594c4eb1a174c0a259d9f17b) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:02:59.740421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c023c01-21f0-4eba-a703-8520c3ca44c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:02:59.741859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c023c01-21f0-4eba-a703-8520c3ca44c8 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:02:59.747464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 591/1200] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:02:59 2022] GET /identity/v3/role_assignments?user.id=a7380dfa594c4eb1a174c0a259d9f17b&scope.project.id=ce5264b4ccd24026a2cdefadb107d54c&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:03:09.593226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=51 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:03:09.744898 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=63 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:03:54.762971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b463d-050b-4d79-9958-e34d0d76004b None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:54.763654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b463d-050b-4d79-9958-e34d0d76004b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:54.763654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b463d-050b-4d79-9958-e34d0d76004b None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:54.764880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 610/1201] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:03:54.770989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f584f7-4ece-4092-bfb8-686de13739e5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:54.771489 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f584f7-4ece-4092-bfb8-686de13739e5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:54.771576 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f584f7-4ece-4092-bfb8-686de13739e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:54.792807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f3f584f7-4ece-4092-bfb8-686de13739e5 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:03:54.794916 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f3f584f7-4ece-4092-bfb8-686de13739e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:03:54.887690 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 592/1202] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:03:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 14 10:03:54.913642 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b463d-050b-4d79-9958-e34d0d76004b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:03:55.053332 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b463d-050b-4d79-9958-e34d0d76004b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:03:55.100649 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e3211-5cc5-4b7e-93c5-74835ebfd670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:03:55.101603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e3211-5cc5-4b7e-93c5-74835ebfd670 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:55.101922 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e3211-5cc5-4b7e-93c5-74835ebfd670 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:55.102199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e3211-5cc5-4b7e-93c5-74835ebfd670 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:55.117181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e3211-5cc5-4b7e-93c5-74835ebfd670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:03:55.119129 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e3211-5cc5-4b7e-93c5-74835ebfd670 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:03:55.140088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-941e3211-5cc5-4b7e-93c5-74835ebfd670 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:03:55.140088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:03:55.140088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:03:55.140088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:03:55.140088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:03:55.140088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:03:55.140088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:03:55.140088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:03:55.140088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:03:55.140088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:03:55.140088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:03:55.140088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:03:55.140088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:03:55.140088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:03:55.140088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:03:55.141186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:03:55.142202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:03:55.142202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:03:55.142202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:03:55.142202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:03:55.142202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 611/1203] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:03:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:03:55.148492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f584f7-4ece-4092-bfb8-686de13739e5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:03:55.284432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f584f7-4ece-4092-bfb8-686de13739e5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:03:55.324875 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:03:55.331364 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188fd7cd-f472-4e0e-ad59-32618544cc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:03:55.332984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fd7cd-f472-4e0e-ad59-32618544cc60 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:55.333955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fd7cd-f472-4e0e-ad59-32618544cc60 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:55.335393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188fd7cd-f472-4e0e-ad59-32618544cc60 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:55.336140 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fd7cd-f472-4e0e-ad59-32618544cc60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:03:55.336486 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fd7cd-f472-4e0e-ad59-32618544cc60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:03:55.337911 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188fd7cd-f472-4e0e-ad59-32618544cc60 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:03:55.350895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 593/1204] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:03:55.357416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e3211-5cc5-4b7e-93c5-74835ebfd670 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:03:55.426607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:03:55.480268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e3211-5cc5-4b7e-93c5-74835ebfd670 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:03:55.521935 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b3050-a08e-48a5-9f3a-faa0db2c7316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:03:55.523057 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b3050-a08e-48a5-9f3a-faa0db2c7316 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:55.523519 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b3050-a08e-48a5-9f3a-faa0db2c7316 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:55.523977 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b3050-a08e-48a5-9f3a-faa0db2c7316 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:55.524960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b3050-a08e-48a5-9f3a-faa0db2c7316 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:03:55.526828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300b3050-a08e-48a5-9f3a-faa0db2c7316 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:03:55.560995 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 612/1205] 10.222.0.61 () {60 vars in 1132 bytes} [Thu Jul 14 10:03:55 2022] POST /identity/v3/users => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:03:57.665193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b4704-cc22-43e2-9550-8ee101fc18b4 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:57.665767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b4704-cc22-43e2-9550-8ee101fc18b4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:57.665767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b4704-cc22-43e2-9550-8ee101fc18b4 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:57.667130 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 594/1206] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:03:57.674409 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d474c65-68c2-4d8a-85cd-516cee65874e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:57.674888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d474c65-68c2-4d8a-85cd-516cee65874e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:57.674976 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d474c65-68c2-4d8a-85cd-516cee65874e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:57.692856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5d474c65-68c2-4d8a-85cd-516cee65874e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:03:57.694078 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5d474c65-68c2-4d8a-85cd-516cee65874e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:03:57.775338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 613/1207] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:03:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:03:57.799243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b4704-cc22-43e2-9550-8ee101fc18b4 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:03:57.931278 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b4704-cc22-43e2-9550-8ee101fc18b4 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:03:57.971976 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1338dec-9a1a-472c-9531-08136fa01a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:03:57.973194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1338dec-9a1a-472c-9531-08136fa01a69 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:57.973717 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1338dec-9a1a-472c-9531-08136fa01a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:57.974193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1338dec-9a1a-472c-9531-08136fa01a69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:57.986712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1338dec-9a1a-472c-9531-08136fa01a69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e1338dec-9a1a-472c-9531-08136fa01a69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:03:57.995990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:03:57.997115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:03:57.998044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:03:57.998044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:03:57.998044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:03:57.998044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:03:57.998044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:03:57.998044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:03:57.998044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:03:57.998044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:03:57.998044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:03:57.998044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:03:57.998044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:03:57.998044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:03:57.998044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 595/1208] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:03:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:03:58.003578 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d474c65-68c2-4d8a-85cd-516cee65874e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:03:58.054371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:03:58.125649 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d474c65-68c2-4d8a-85cd-516cee65874e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:03:58.165974 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f22269-68b7-4e5f-9337-fb67d7330d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:03:58.167220 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f22269-68b7-4e5f-9337-fb67d7330d52 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:58.167715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f22269-68b7-4e5f-9337-fb67d7330d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:58.168448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f22269-68b7-4e5f-9337-fb67d7330d52 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:58.169439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f22269-68b7-4e5f-9337-fb67d7330d52 None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:03:58.170074 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f22269-68b7-4e5f-9337-fb67d7330d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:03:58.171514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f22269-68b7-4e5f-9337-fb67d7330d52 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:03:58.187095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 614/1209] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:03:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:03:58.188493 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1338dec-9a1a-472c-9531-08136fa01a69 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:03:58.252371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:03:58.310838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1338dec-9a1a-472c-9531-08136fa01a69 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:03:58.348320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a160661-7bd1-4263-8e61-c404bf55aa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:03:58.349793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a160661-7bd1-4263-8e61-c404bf55aa65 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:58.349943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a160661-7bd1-4263-8e61-c404bf55aa65 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:58.350220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a160661-7bd1-4263-8e61-c404bf55aa65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:58.361556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a160661-7bd1-4263-8e61-c404bf55aa65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:03:58.363234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a160661-7bd1-4263-8e61-c404bf55aa65 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:03:58.385283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-4a160661-7bd1-4263-8e61-c404bf55aa65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:03:58.385283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:03:58.385283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:03:58.385283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:03:58.385283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:03:58.385283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:03:58.385283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:03:58.385283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:03:58.385283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:03:58.385283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:03:58.385283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:03:58.385283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:03:58.385283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:03:58.385283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:03:58.385283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:03:58.386309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:03:58.387223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:03:58.387223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:03:58.387223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:03:58.387223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:03:58.387223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 596/1210] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:03:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:03:58.393221 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f22269-68b7-4e5f-9337-fb67d7330d52 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:03:58.503627 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:03:58.510755 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f22269-68b7-4e5f-9337-fb67d7330d52 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:03:58.554227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7d387-2232-4d22-9922-d8fb1e64e6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:03:58.555305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7d387-2232-4d22-9922-d8fb1e64e6ce None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:58.555811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7d387-2232-4d22-9922-d8fb1e64e6ce None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:58.556188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7d387-2232-4d22-9922-d8fb1e64e6ce None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:58.557116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7d387-2232-4d22-9922-d8fb1e64e6ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:03:58.557523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7d387-2232-4d22-9922-d8fb1e64e6ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:03:58.559010 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7d387-2232-4d22-9922-d8fb1e64e6ce None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:03:58.578209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 615/1211] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:03:58.578432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a160661-7bd1-4263-8e61-c404bf55aa65 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:03:58.691483 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a160661-7bd1-4263-8e61-c404bf55aa65 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:03:58.693128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:03:58.733800 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709c309-cf23-4de2-b0d0-a5b0067831ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:03:58.734934 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709c309-cf23-4de2-b0d0-a5b0067831ac None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:58.735210 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709c309-cf23-4de2-b0d0-a5b0067831ac None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:58.735448 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5709c309-cf23-4de2-b0d0-a5b0067831ac None admin] PATH_INFO: `/v3/users/placement` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:58.743230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709c309-cf23-4de2-b0d0-a5b0067831ac None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:03:58.744658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5709c309-cf23-4de2-b0d0-a5b0067831ac None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-5709c309-cf23-4de2-b0d0-a5b0067831ac None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:03:58.760100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:03:58.761332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:03:58.762515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:03:58.762515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:03:58.762515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:03:58.762515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:03:58.762515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:03:58.762515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:03:58.762515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:03:58.762515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:03:58.762515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:03:58.762515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:03:58.762515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 597/1212] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:03:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:03:58.767089 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7d387-2232-4d22-9922-d8fb1e64e6ce None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:03:58.885228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7d387-2232-4d22-9922-d8fb1e64e6ce None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:03:58.923389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca0c3f-30c2-44f6-9235-f9909c9f5f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:03:58.924060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca0c3f-30c2-44f6-9235-f9909c9f5f58 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:58.924336 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca0c3f-30c2-44f6-9235-f9909c9f5f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:58.924608 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca0c3f-30c2-44f6-9235-f9909c9f5f58 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:58.925494 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca0c3f-30c2-44f6-9235-f9909c9f5f58 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:03:58.925809 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca0c3f-30c2-44f6-9235-f9909c9f5f58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:03:58.927279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca0c3f-30c2-44f6-9235-f9909c9f5f58 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:03:58.951828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 616/1213] 10.222.0.61 () {56 vars in 1143 bytes} [Thu Jul 14 10:03:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:03:58.953446 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709c309-cf23-4de2-b0d0-a5b0067831ac None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:03:59.080152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5709c309-cf23-4de2-b0d0-a5b0067831ac None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:03:59.121677 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d5b2f-1313-4d2a-ac9b-57bdd0102d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:03:59.122503 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d5b2f-1313-4d2a-ac9b-57bdd0102d01 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:59.122780 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d5b2f-1313-4d2a-ac9b-57bdd0102d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:59.123175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d5b2f-1313-4d2a-ac9b-57bdd0102d01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:59.161330 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:03:59.210803 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d5b2f-1313-4d2a-ac9b-57bdd0102d01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:03:59.212402 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d5b2f-1313-4d2a-ac9b-57bdd0102d01 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:03:59.340083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 598/1214] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:03:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:03:59.346168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca0c3f-30c2-44f6-9235-f9909c9f5f58 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:03:59.366859 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:03:59.471974 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca0c3f-30c2-44f6-9235-f9909c9f5f58 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:03:59.518557 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c506693-0f0d-4f53-b73b-85c4535dc7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:03:59.519421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c506693-0f0d-4f53-b73b-85c4535dc7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:59.519710 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c506693-0f0d-4f53-b73b-85c4535dc7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:59.519984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c506693-0f0d-4f53-b73b-85c4535dc7c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:59.533843 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c506693-0f0d-4f53-b73b-85c4535dc7c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:03:59.536094 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c506693-0f0d-4f53-b73b-85c4535dc7c6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:03:59.558740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-7c506693-0f0d-4f53-b73b-85c4535dc7c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:03:59.558740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:03:59.558740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:03:59.558740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:03:59.558740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:03:59.558740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:03:59.558740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:03:59.558740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:03:59.558740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:03:59.558740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:03:59.558740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:03:59.558740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:03:59.558740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:03:59.558740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:03:59.558740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:03:59.560112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:03:59.565743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:03:59.565743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:03:59.565743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:03:59.565743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:03:59.565743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 617/1215] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:03:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:03:59.567033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d5b2f-1313-4d2a-ac9b-57bdd0102d01 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:03:59.639891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:03:59.686647 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d5b2f-1313-4d2a-ac9b-57bdd0102d01 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:03:59.725935 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569ba25-75cf-4192-b528-3e7d2ef5a7ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:03:59.726619 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569ba25-75cf-4192-b528-3e7d2ef5a7ba None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:59.726860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569ba25-75cf-4192-b528-3e7d2ef5a7ba None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:59.727224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6569ba25-75cf-4192-b528-3e7d2ef5a7ba None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:59.727856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569ba25-75cf-4192-b528-3e7d2ef5a7ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:03:59.728179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569ba25-75cf-4192-b528-3e7d2ef5a7ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:03:59.729569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6569ba25-75cf-4192-b528-3e7d2ef5a7ba None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:03:59.746300 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 599/1216] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:03:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:03:59.747568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c506693-0f0d-4f53-b73b-85c4535dc7c6 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:03:59.822756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:03:59.856523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c506693-0f0d-4f53-b73b-85c4535dc7c6 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:03:59.894160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32c56d-f8bd-46eb-a823-bfde342760ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:03:59.895045 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32c56d-f8bd-46eb-a823-bfde342760ea None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:03:59.895785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32c56d-f8bd-46eb-a823-bfde342760ea None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:03:59.896154 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32c56d-f8bd-46eb-a823-bfde342760ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:03:59.903925 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32c56d-f8bd-46eb-a823-bfde342760ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:03:59.905560 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32c56d-f8bd-46eb-a823-bfde342760ea None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:03:59.920978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-fe32c56d-f8bd-46eb-a823-bfde342760ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:03:59.920978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:03:59.920978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:03:59.920978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:03:59.920978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:03:59.920978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:03:59.920978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:03:59.920978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:03:59.920978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:03:59.920978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:03:59.920978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:03:59.920978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:03:59.920978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:03:59.920978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:03:59.920978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:03:59.922118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:03:59.923251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:03:59.923251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:03:59.923251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:03:59.923251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:03:59.923251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:03:59.923251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:03:59.923251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:03:59.923251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:03:59.930173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 618/1217] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:03:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:03:59.931265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569ba25-75cf-4192-b528-3e7d2ef5a7ba None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:00.050903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6569ba25-75cf-4192-b528-3e7d2ef5a7ba None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:00.078017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:00.092843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dff7e6-c7fd-4682-9daf-927898fb479c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:00.093834 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dff7e6-c7fd-4682-9daf-927898fb479c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:00.094217 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dff7e6-c7fd-4682-9daf-927898fb479c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:00.094648 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dff7e6-c7fd-4682-9daf-927898fb479c None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:00.095583 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dff7e6-c7fd-4682-9daf-927898fb479c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:00.096011 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dff7e6-c7fd-4682-9daf-927898fb479c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:00.097613 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dff7e6-c7fd-4682-9daf-927898fb479c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:00.113163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 600/1218] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:03:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:04:00.117896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32c56d-f8bd-46eb-a823-bfde342760ea None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:00.239714 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32c56d-f8bd-46eb-a823-bfde342760ea None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:00.273277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:00.281646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900b86cd-5845-405d-8115-5657327df77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:00.282614 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b86cd-5845-405d-8115-5657327df77f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:00.283109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b86cd-5845-405d-8115-5657327df77f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:00.283473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900b86cd-5845-405d-8115-5657327df77f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:00.284452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b86cd-5845-405d-8115-5657327df77f None admin] RBAC: Adding query filter params (role.id=ba4088a289ea4b9a9aa9d9cd7a27e731, scope.project.id=57662256880f43018039715a755c087c, user.id=ff4c42a4f4f549378e5bc1cd92f801c7) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:00.284772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b86cd-5845-405d-8115-5657327df77f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:00.286917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900b86cd-5845-405d-8115-5657327df77f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:00.293861 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 619/1219] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:04:00 2022] GET /identity/v3/role_assignments?user.id=ff4c42a4f4f549378e5bc1cd92f801c7&scope.project.id=57662256880f43018039715a755c087c&role.id=ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:02.421156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe954d-ef6d-4b01-a385-654a5a7c664e None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:02.421689 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe954d-ef6d-4b01-a385-654a5a7c664e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:02.421779 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe954d-ef6d-4b01-a385-654a5a7c664e None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:02.423185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 601/1220] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:04:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:04:02.431242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716c726-f019-4c7d-8d86-4cf518e85737 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:02.431242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716c726-f019-4c7d-8d86-4cf518e85737 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:02.431242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5716c726-f019-4c7d-8d86-4cf518e85737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:02.449059 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5716c726-f019-4c7d-8d86-4cf518e85737 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:04:02.450257 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5716c726-f019-4c7d-8d86-4cf518e85737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:04:02.522221 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 620/1221] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:04:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:04:02.545940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe954d-ef6d-4b01-a385-654a5a7c664e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:02.655950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe954d-ef6d-4b01-a385-654a5a7c664e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:02.690099 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d57ee-1914-4968-8cdb-f43775ebd252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:02.690886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d57ee-1914-4968-8cdb-f43775ebd252 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:02.691166 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d57ee-1914-4968-8cdb-f43775ebd252 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:02.691437 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d57ee-1914-4968-8cdb-f43775ebd252 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:02.703600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d57ee-1914-4968-8cdb-f43775ebd252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-bc6d57ee-1914-4968-8cdb-f43775ebd252 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:04:02.712372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:02.713711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:02.714902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:02.714902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:02.714902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:02.714902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:02.714902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:04:02.714902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:04:02.714902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:04:02.714902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:04:02.714902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:04:02.714902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:04:02.714902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:04:02.714902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:02.714902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 602/1222] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:04:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:02.720110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716c726-f019-4c7d-8d86-4cf518e85737 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:02.835353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5716c726-f019-4c7d-8d86-4cf518e85737 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:02.859554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:02.871485 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c712d4-d41b-4bfd-94d8-8f1764000532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:02.872230 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c712d4-d41b-4bfd-94d8-8f1764000532 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:02.872506 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c712d4-d41b-4bfd-94d8-8f1764000532 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:02.872783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c712d4-d41b-4bfd-94d8-8f1764000532 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:02.873638 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c712d4-d41b-4bfd-94d8-8f1764000532 None admin] RBAC: Adding query filter params (name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:02.873965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c712d4-d41b-4bfd-94d8-8f1764000532 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:02.875509 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c712d4-d41b-4bfd-94d8-8f1764000532 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:02.885488 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 621/1223] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:04:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:04:02.891015 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d57ee-1914-4968-8cdb-f43775ebd252 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:02.937379 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:02.999914 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d57ee-1914-4968-8cdb-f43775ebd252 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:03.047157 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a7c00-d2ed-42ef-9df0-ad59ed3e6b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:03.047957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a7c00-d2ed-42ef-9df0-ad59ed3e6b09 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:03.048235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a7c00-d2ed-42ef-9df0-ad59ed3e6b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:03.048506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a7c00-d2ed-42ef-9df0-ad59ed3e6b09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:03.060085 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a7c00-d2ed-42ef-9df0-ad59ed3e6b09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:03.061575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a7c00-d2ed-42ef-9df0-ad59ed3e6b09 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:03.078559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-bd0a7c00-d2ed-42ef-9df0-ad59ed3e6b09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:03.078559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:03.078559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:03.078559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:03.078559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:03.078559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:03.078559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:03.078559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:03.078559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:03.078559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:03.078559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:03.078559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:03.078559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:04:03.078559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:04:03.078559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:03.079694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:03.080822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:04:03.080822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:04:03.080822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:03.080822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:03.084651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 603/1224] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:04:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:03.085881 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c712d4-d41b-4bfd-94d8-8f1764000532 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:03.199532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c712d4-d41b-4bfd-94d8-8f1764000532 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:03.237374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c437f-550e-4d3d-83a9-0f6a86ebba44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:03.238279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c437f-550e-4d3d-83a9-0f6a86ebba44 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:03.238719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c437f-550e-4d3d-83a9-0f6a86ebba44 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:03.239158 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c437f-550e-4d3d-83a9-0f6a86ebba44 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:03.240051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c437f-550e-4d3d-83a9-0f6a86ebba44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:03.240352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c437f-550e-4d3d-83a9-0f6a86ebba44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:03.241733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c437f-550e-4d3d-83a9-0f6a86ebba44 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:03.254485 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 622/1225] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:03.261414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a7c00-d2ed-42ef-9df0-ad59ed3e6b09 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:03.365351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:03.375476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a7c00-d2ed-42ef-9df0-ad59ed3e6b09 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:03.409602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474d4ba-d0e2-4be0-9412-1feea44b0442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:03.410403 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474d4ba-d0e2-4be0-9412-1feea44b0442 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:03.410687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474d4ba-d0e2-4be0-9412-1feea44b0442 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:03.410999 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0474d4ba-d0e2-4be0-9412-1feea44b0442 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:03.419905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474d4ba-d0e2-4be0-9412-1feea44b0442 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:03.421441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0474d4ba-d0e2-4be0-9412-1feea44b0442 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-0474d4ba-d0e2-4be0-9412-1feea44b0442 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:04:03.436543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:03.437756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:03.439121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:03.439121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:03.439121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:04:03.439121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:04:03.439121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:04:03.439121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:04:03.439121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:04:03.439121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:04:03.439121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:04:03.439121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:03.439121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 604/1226] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:04:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:04:03.443821 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c437f-550e-4d3d-83a9-0f6a86ebba44 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:03.490410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:03.544752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c437f-550e-4d3d-83a9-0f6a86ebba44 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:03.579074 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515e557-a5dc-434c-8c21-898d0830a955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:03.579741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e557-a5dc-434c-8c21-898d0830a955 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:03.580019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e557-a5dc-434c-8c21-898d0830a955 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:03.580289 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5515e557-a5dc-434c-8c21-898d0830a955 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:03.581164 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515e557-a5dc-434c-8c21-898d0830a955 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:03.581464 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515e557-a5dc-434c-8c21-898d0830a955 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:03.583467 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5515e557-a5dc-434c-8c21-898d0830a955 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:03.601793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 623/1227] 10.222.0.61 () {56 vars in 1143 bytes} [Thu Jul 14 10:04:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:03.608542 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474d4ba-d0e2-4be0-9412-1feea44b0442 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:03.729426 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0474d4ba-d0e2-4be0-9412-1feea44b0442 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:03.765360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ee6e1-ef6d-4c0a-863d-81357f44a491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:03.766156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ee6e1-ef6d-4c0a-863d-81357f44a491 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:03.766436 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ee6e1-ef6d-4c0a-863d-81357f44a491 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:03.766708 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3ee6e1-ef6d-4c0a-863d-81357f44a491 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:03.780511 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ee6e1-ef6d-4c0a-863d-81357f44a491 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:03.782023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3ee6e1-ef6d-4c0a-863d-81357f44a491 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:03.799716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-7f3ee6e1-ef6d-4c0a-863d-81357f44a491 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:03.799716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:03.799716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:03.799716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:03.799716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:03.799716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:03.799716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:03.799716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:03.799716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:03.799716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:03.799716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:03.799716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:03.799716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:04:03.799716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:04:03.799716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:03.800748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:03.801885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:04:03.801885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:04:03.801885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:03.801885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:03.801885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 605/1228] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:04:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:03.806652 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515e557-a5dc-434c-8c21-898d0830a955 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:03.925203 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5515e557-a5dc-434c-8c21-898d0830a955 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:03.928841 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:03.965065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e3cef-d011-4050-9bde-42e1139473a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:03.965811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e3cef-d011-4050-9bde-42e1139473a5 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:03.966448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e3cef-d011-4050-9bde-42e1139473a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:03.966735 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e3cef-d011-4050-9bde-42e1139473a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:03.967665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e3cef-d011-4050-9bde-42e1139473a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:03.968096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e3cef-d011-4050-9bde-42e1139473a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:03.969916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e3cef-d011-4050-9bde-42e1139473a5 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:03.984025 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 624/1229] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:04:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:03.990407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ee6e1-ef6d-4c0a-863d-81357f44a491 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:04.000034 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:04.103869 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3ee6e1-ef6d-4c0a-863d-81357f44a491 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:04.142595 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a1ab2-4a40-4aca-8871-c1698d0995c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:04.143877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a1ab2-4a40-4aca-8871-c1698d0995c3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:04.144272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a1ab2-4a40-4aca-8871-c1698d0995c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:04.144724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a1ab2-4a40-4aca-8871-c1698d0995c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:04.153316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a1ab2-4a40-4aca-8871-c1698d0995c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:04.154906 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a1ab2-4a40-4aca-8871-c1698d0995c3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:04.173097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-0c3a1ab2-4a40-4aca-8871-c1698d0995c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:04:04.173097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:04.173097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:04.173097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:04.173097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:04.173097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:04.173097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:04.173097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:04.173097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:04.173097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:04.173097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:04.173097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:04.173097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:04:04.173097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:04:04.173097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:04.174316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:04.175314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:04:04.175314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:04:04.175314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:04:04.175314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:04:04.175314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:04:04.175314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:04:04.175314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:04:04.175314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:04.175314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 606/1230] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:04:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:04.181422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e3cef-d011-4050-9bde-42e1139473a5 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:04.298613 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e3cef-d011-4050-9bde-42e1139473a5 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:04.338556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644fe4d-02d0-486f-b1be-62932e5d3788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:04.339362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644fe4d-02d0-486f-b1be-62932e5d3788 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:04.339648 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644fe4d-02d0-486f-b1be-62932e5d3788 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:04.339929 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644fe4d-02d0-486f-b1be-62932e5d3788 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:04.340876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644fe4d-02d0-486f-b1be-62932e5d3788 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:04.341221 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644fe4d-02d0-486f-b1be-62932e5d3788 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:04.342909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644fe4d-02d0-486f-b1be-62932e5d3788 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:04.358175 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 625/1231] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:04:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:04.366124 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a1ab2-4a40-4aca-8871-c1698d0995c3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:04.435334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:04.474141 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a1ab2-4a40-4aca-8871-c1698d0995c3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:04.513682 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdfc88c-813a-4553-a451-7b01ddbefc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:04.514494 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfc88c-813a-4553-a451-7b01ddbefc1d None admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:04.514770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfc88c-813a-4553-a451-7b01ddbefc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:04.515041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdfc88c-813a-4553-a451-7b01ddbefc1d None admin] PATH_INFO: `/v3/projects/57662256880f43018039715a755c087c/users/ff4c42a4f4f549378e5bc1cd92f801c7/roles/ba4088a289ea4b9a9aa9d9cd7a27e731` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:04.531189 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfc88c-813a-4553-a451-7b01ddbefc1d None admin] RBAC: Authorizing `identity:create_grant(project_id=57662256880f43018039715a755c087c, user_id=ff4c42a4f4f549378e5bc1cd92f801c7, role_id=ba4088a289ea4b9a9aa9d9cd7a27e731)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:04.532684 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdfc88c-813a-4553-a451-7b01ddbefc1d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:04.545980 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 607/1232] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:04:04 2022] PUT /identity/v3/projects/57662256880f43018039715a755c087c/users/ff4c42a4f4f549378e5bc1cd92f801c7/roles/ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:04:06.650449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44de8d-1464-4b0c-b304-96867d798059 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:06.650973 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44de8d-1464-4b0c-b304-96867d798059 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:06.651156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c44de8d-1464-4b0c-b304-96867d798059 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:06.653234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 626/1233] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:04:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:04:06.659274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc9e60-4556-451a-bfe4-d03193b3fc23 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:06.659724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc9e60-4556-451a-bfe4-d03193b3fc23 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:06.659921 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dc9e60-4556-451a-bfe4-d03193b3fc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:06.678269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-28dc9e60-4556-451a-bfe4-d03193b3fc23 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:04:06.679596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-28dc9e60-4556-451a-bfe4-d03193b3fc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:04:06.748347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 608/1234] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:04:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:04:06.773019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44de8d-1464-4b0c-b304-96867d798059 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:06.799472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:06.880767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c44de8d-1464-4b0c-b304-96867d798059 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:06.916311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f0772-e2de-4a54-a9e2-9ed9e31cad32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:06.917199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f0772-e2de-4a54-a9e2-9ed9e31cad32 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:06.917489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f0772-e2de-4a54-a9e2-9ed9e31cad32 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:06.917793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f0772-e2de-4a54-a9e2-9ed9e31cad32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:06.929684 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f0772-e2de-4a54-a9e2-9ed9e31cad32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-768f0772-e2de-4a54-a9e2-9ed9e31cad32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:04:06.938440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:06.939694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:06.946489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:06.946489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:06.946489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:06.946489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:06.946489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:04:06.946489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:04:06.946489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:04:06.946489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:04:06.946489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:04:06.946489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:04:06.946489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 14 10:04:06.946489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:04:06.946489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 627/1235] 10.222.0.61 () {56 vars in 1102 bytes} [Thu Jul 14 10:04:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:04:06.947362 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc9e60-4556-451a-bfe4-d03193b3fc23 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:06.997262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:07.072658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dc9e60-4556-451a-bfe4-d03193b3fc23 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:07.112399 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56628c53-2082-4b95-ad69-60c688ba00f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:07.113175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56628c53-2082-4b95-ad69-60c688ba00f3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:07.113558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56628c53-2082-4b95-ad69-60c688ba00f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:07.113875 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56628c53-2082-4b95-ad69-60c688ba00f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:07.114684 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56628c53-2082-4b95-ad69-60c688ba00f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:07.114991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56628c53-2082-4b95-ad69-60c688ba00f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:07.117078 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56628c53-2082-4b95-ad69-60c688ba00f3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:07.126158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 609/1236] 10.222.0.61 () {56 vars in 1103 bytes} [Thu Jul 14 10:04:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:07.133682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f0772-e2de-4a54-a9e2-9ed9e31cad32 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:07.238148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:07.255607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f0772-e2de-4a54-a9e2-9ed9e31cad32 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:07.291625 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c575a79-8b51-4868-a547-c3a8c30cff10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:07.292452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c575a79-8b51-4868-a547-c3a8c30cff10 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:07.292745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c575a79-8b51-4868-a547-c3a8c30cff10 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:07.293301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c575a79-8b51-4868-a547-c3a8c30cff10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:07.304294 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c575a79-8b51-4868-a547-c3a8c30cff10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:07.305763 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c575a79-8b51-4868-a547-c3a8c30cff10 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:07.324377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-4c575a79-8b51-4868-a547-c3a8c30cff10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:07.324377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:07.324377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:07.324377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:07.324377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:07.324377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:07.324377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:07.324377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:07.324377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:07.324377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:07.324377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:07.324377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:07.324377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:04:07.324377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:04:07.324377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:07.325637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:07.326571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:04:07.326571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:04:07.326571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:07.326571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:04:07.326571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 628/1237] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:04:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:07.331579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56628c53-2082-4b95-ad69-60c688ba00f3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:07.435851 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56628c53-2082-4b95-ad69-60c688ba00f3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:07.445606 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:07.471458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478577c6-f69a-472a-b240-6ac86a61c526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:07.472126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478577c6-f69a-472a-b240-6ac86a61c526 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:07.472400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478577c6-f69a-472a-b240-6ac86a61c526 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:07.472673 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478577c6-f69a-472a-b240-6ac86a61c526 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:07.473463 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478577c6-f69a-472a-b240-6ac86a61c526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:07.473771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478577c6-f69a-472a-b240-6ac86a61c526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:07.475137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478577c6-f69a-472a-b240-6ac86a61c526 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:07.487146 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 610/1238] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:04:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:07.493331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c575a79-8b51-4868-a547-c3a8c30cff10 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:07.603303 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c575a79-8b51-4868-a547-c3a8c30cff10 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:07.639274 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f21e1f-6a84-4e51-ab81-1e62129a3871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:07.640065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f21e1f-6a84-4e51-ab81-1e62129a3871 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:07.640339 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f21e1f-6a84-4e51-ab81-1e62129a3871 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:07.640610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f21e1f-6a84-4e51-ab81-1e62129a3871 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:07.643439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:07.648791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f21e1f-6a84-4e51-ab81-1e62129a3871 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:07.650357 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f21e1f-6a84-4e51-ab81-1e62129a3871 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-98f21e1f-6a84-4e51-ab81-1e62129a3871 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:04:07.665680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:07.667362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:07.668593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:07.668593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:07.668593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:04:07.668593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:04:07.668593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:04:07.668593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:04:07.668593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:04:07.668593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:04:07.668593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:04:07.668593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:04:07.668593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 629/1239] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:04:07 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:07.673047 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478577c6-f69a-472a-b240-6ac86a61c526 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:07.784006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478577c6-f69a-472a-b240-6ac86a61c526 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:07.821880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21f3c6-c610-4f3f-b0fb-01114b327d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:07.822657 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21f3c6-c610-4f3f-b0fb-01114b327d86 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:07.822974 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21f3c6-c610-4f3f-b0fb-01114b327d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:07.823285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b21f3c6-c610-4f3f-b0fb-01114b327d86 None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:07.824276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21f3c6-c610-4f3f-b0fb-01114b327d86 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:07.824614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21f3c6-c610-4f3f-b0fb-01114b327d86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:07.826306 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b21f3c6-c610-4f3f-b0fb-01114b327d86 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:07.845521 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 611/1240] 10.222.0.61 () {56 vars in 1143 bytes} [Thu Jul 14 10:04:07 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:07.852424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f21e1f-6a84-4e51-ab81-1e62129a3871 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:07.967945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f21e1f-6a84-4e51-ab81-1e62129a3871 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:08.003836 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be598357-6a14-4368-b417-7e02c11e8d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:08.004601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be598357-6a14-4368-b417-7e02c11e8d26 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:08.004971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be598357-6a14-4368-b417-7e02c11e8d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:08.005251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be598357-6a14-4368-b417-7e02c11e8d26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:08.074267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:08.081897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be598357-6a14-4368-b417-7e02c11e8d26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:08.084113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be598357-6a14-4368-b417-7e02c11e8d26 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:08.196348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 630/1241] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:04:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:04:08.202779 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21f3c6-c610-4f3f-b0fb-01114b327d86 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:08.237123 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:08.321611 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b21f3c6-c610-4f3f-b0fb-01114b327d86 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:08.363463 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9127d18-e7d6-4bc9-b742-81d000609142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:08.364831 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9127d18-e7d6-4bc9-b742-81d000609142 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:08.365112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9127d18-e7d6-4bc9-b742-81d000609142 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:08.365381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9127d18-e7d6-4bc9-b742-81d000609142 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:08.376481 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9127d18-e7d6-4bc9-b742-81d000609142 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:08.377994 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9127d18-e7d6-4bc9-b742-81d000609142 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:08.398959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-e9127d18-e7d6-4bc9-b742-81d000609142 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:08.398959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:08.398959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:08.398959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:08.398959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:08.398959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:08.398959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:08.398959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:08.398959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:08.398959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:08.398959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:08.398959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:08.398959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:04:08.398959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:04:08.398959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:08.400128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:08.401283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:04:08.401283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:04:08.401283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:08.401283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:08.406024 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 612/1242] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:04:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:08.407460 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be598357-6a14-4368-b417-7e02c11e8d26 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:08.525456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:08.532523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be598357-6a14-4368-b417-7e02c11e8d26 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:08.570345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c063e-3315-4062-812e-8ae733bd8062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:08.571039 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c063e-3315-4062-812e-8ae733bd8062 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:08.571288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c063e-3315-4062-812e-8ae733bd8062 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:08.571561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c063e-3315-4062-812e-8ae733bd8062 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:08.572289 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c063e-3315-4062-812e-8ae733bd8062 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:08.572588 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c063e-3315-4062-812e-8ae733bd8062 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:08.574001 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c063e-3315-4062-812e-8ae733bd8062 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:08.586886 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 631/1243] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:04:08.592898 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9127d18-e7d6-4bc9-b742-81d000609142 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:08.682759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:08.712677 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9127d18-e7d6-4bc9-b742-81d000609142 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:08.756792 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f519c88-be35-45fc-b263-37d7aeee5fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:08.758275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f519c88-be35-45fc-b263-37d7aeee5fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:08.758742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f519c88-be35-45fc-b263-37d7aeee5fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:08.759101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f519c88-be35-45fc-b263-37d7aeee5fc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:08.766255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f519c88-be35-45fc-b263-37d7aeee5fc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:08.767688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f519c88-be35-45fc-b263-37d7aeee5fc5 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:08.782793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-7f519c88-be35-45fc-b263-37d7aeee5fc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:04:08.782793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:08.782793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:08.782793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:08.782793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:08.782793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:08.782793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:08.782793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:08.782793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:08.782793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:08.782793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:08.782793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:08.782793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:04:08.782793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:04:08.782793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:08.783908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:08.785293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:04:08.785293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:04:08.785293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:04:08.785293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:04:08.785293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:04:08.785293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:04:08.785293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:04:08.785293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:08.785293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 613/1244] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:04:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:08.790642 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c063e-3315-4062-812e-8ae733bd8062 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:08.899821 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c063e-3315-4062-812e-8ae733bd8062 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:08.935800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0f625-2fcf-4e44-b42e-af2e6b42bbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:08.936396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0f625-2fcf-4e44-b42e-af2e6b42bbb6 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:08.936683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0f625-2fcf-4e44-b42e-af2e6b42bbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:08.937018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0f625-2fcf-4e44-b42e-af2e6b42bbb6 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:08.937767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0f625-2fcf-4e44-b42e-af2e6b42bbb6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:08.938075 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0f625-2fcf-4e44-b42e-af2e6b42bbb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:08.939502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0f625-2fcf-4e44-b42e-af2e6b42bbb6 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:08.953011 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 632/1245] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:04:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:08.959150 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f519c88-be35-45fc-b263-37d7aeee5fc5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:09.079099 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f519c88-be35-45fc-b263-37d7aeee5fc5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:09.119204 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a0b474-a9d6-4b48-a4e5-de80c3fdf0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:09.119932 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0b474-a9d6-4b48-a4e5-de80c3fdf0ac None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:09.120281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0b474-a9d6-4b48-a4e5-de80c3fdf0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:09.120624 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a0b474-a9d6-4b48-a4e5-de80c3fdf0ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:09.121601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0b474-a9d6-4b48-a4e5-de80c3fdf0ac None admin] RBAC: Adding query filter params (role.id=ba4088a289ea4b9a9aa9d9cd7a27e731, scope.project.id=57662256880f43018039715a755c087c, user.id=ff4c42a4f4f549378e5bc1cd92f801c7) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:09.122057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0b474-a9d6-4b48-a4e5-de80c3fdf0ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:09.123622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a0b474-a9d6-4b48-a4e5-de80c3fdf0ac None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:09.130751 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 614/1246] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:04:08 2022] GET /identity/v3/role_assignments?user.id=ff4c42a4f4f549378e5bc1cd92f801c7&scope.project.id=57662256880f43018039715a755c087c&role.id=ba4088a289ea4b9a9aa9d9cd7a27e731 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:11.275742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fd4c7-6cd2-44f4-af3e-155e54477ba4 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:11.276287 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fd4c7-6cd2-44f4-af3e-155e54477ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:11.276287 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fd4c7-6cd2-44f4-af3e-155e54477ba4 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:11.277986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 633/1247] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:04:11.283265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fac624-7ebf-45b7-89a0-b9597782e3ce None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:11.283722 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fac624-7ebf-45b7-89a0-b9597782e3ce None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:11.283810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fac624-7ebf-45b7-89a0-b9597782e3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:11.289647 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:11.303789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-64fac624-7ebf-45b7-89a0-b9597782e3ce None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:04:11.305108 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-64fac624-7ebf-45b7-89a0-b9597782e3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:04:11.380406 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 615/1248] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:04:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:04:11.405306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fd4c7-6cd2-44f4-af3e-155e54477ba4 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:11.425113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:11.519358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fd4c7-6cd2-44f4-af3e-155e54477ba4 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:11.556529 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b9ef0-56b2-42c8-abf4-532d4bd8d625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:11.557565 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b9ef0-56b2-42c8-abf4-532d4bd8d625 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:11.558000 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b9ef0-56b2-42c8-abf4-532d4bd8d625 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:11.558424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42b9ef0-56b2-42c8-abf4-532d4bd8d625 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:11.573081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42b9ef0-56b2-42c8-abf4-532d4bd8d625 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-c42b9ef0-56b2-42c8-abf4-532d4bd8d625 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:04:11.583190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:11.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:11.585579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:11.585579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:11.585579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:11.585579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:11.585579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:04:11.585579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:04:11.585579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:04:11.585579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:04:11.585579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:04:11.585579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:04:11.585579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:04:11.585579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:04:11.585579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 634/1249] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:04:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:11.591220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fac624-7ebf-45b7-89a0-b9597782e3ce None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:11.712243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fac624-7ebf-45b7-89a0-b9597782e3ce None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:11.746700 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:11.752732 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d91e57-db76-49dc-895e-1dbbf304d52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:11.753589 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d91e57-db76-49dc-895e-1dbbf304d52c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:11.753896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d91e57-db76-49dc-895e-1dbbf304d52c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:11.754196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d91e57-db76-49dc-895e-1dbbf304d52c None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:11.755072 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d91e57-db76-49dc-895e-1dbbf304d52c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:11.755436 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d91e57-db76-49dc-895e-1dbbf304d52c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:11.756987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d91e57-db76-49dc-895e-1dbbf304d52c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:11.766276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 616/1250] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:04:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:11.772690 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b9ef0-56b2-42c8-abf4-532d4bd8d625 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:11.857685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:11.883320 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42b9ef0-56b2-42c8-abf4-532d4bd8d625 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:11.920407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448a59e-f5a3-462c-97bd-0eb6b76e480c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:11.921267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448a59e-f5a3-462c-97bd-0eb6b76e480c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:11.921549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448a59e-f5a3-462c-97bd-0eb6b76e480c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:11.921814 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a448a59e-f5a3-462c-97bd-0eb6b76e480c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:11.933982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448a59e-f5a3-462c-97bd-0eb6b76e480c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:11.935471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a448a59e-f5a3-462c-97bd-0eb6b76e480c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:11.954672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-a448a59e-f5a3-462c-97bd-0eb6b76e480c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:11.954672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:11.954672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:11.954672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:11.954672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:11.954672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:11.954672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:11.954672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:11.954672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:11.954672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:11.954672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:11.954672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:11.954672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:04:11.954672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:04:11.954672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:11.955851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:11.962191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:04:11.962191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:04:11.962191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:11.962191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:04:11.962191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 635/1251] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:04:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 14 10:04:11.963297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d91e57-db76-49dc-895e-1dbbf304d52c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:12.079805 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d91e57-db76-49dc-895e-1dbbf304d52c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:12.120712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1b114-307f-4510-bec7-331099a9cc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:12.121527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1b114-307f-4510-bec7-331099a9cc75 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:12.121848 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1b114-307f-4510-bec7-331099a9cc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:12.122184 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a1b114-307f-4510-bec7-331099a9cc75 None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:12.122995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1b114-307f-4510-bec7-331099a9cc75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:12.123357 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1b114-307f-4510-bec7-331099a9cc75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:12.124756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a1b114-307f-4510-bec7-331099a9cc75 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:12.139474 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 617/1252] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:12.145799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448a59e-f5a3-462c-97bd-0eb6b76e480c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:12.253371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a448a59e-f5a3-462c-97bd-0eb6b76e480c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:12.281128 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:12.289418 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416218b-6aad-40af-a0eb-96eab86f504f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:12.290539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416218b-6aad-40af-a0eb-96eab86f504f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:12.290784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416218b-6aad-40af-a0eb-96eab86f504f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:12.291088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416218b-6aad-40af-a0eb-96eab86f504f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:12.300207 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416218b-6aad-40af-a0eb-96eab86f504f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:12.302629 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416218b-6aad-40af-a0eb-96eab86f504f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-2416218b-6aad-40af-a0eb-96eab86f504f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:04:12.319019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:12.320297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:12.321490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:12.321490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:12.321490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:04:12.321490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:04:12.321490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:04:12.321490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:04:12.321490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:04:12.321490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:04:12.321490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:04:12.321490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:04:12.321490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 636/1253] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:04:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:12.326327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1b114-307f-4510-bec7-331099a9cc75 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:12.385209 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:12.436392 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a1b114-307f-4510-bec7-331099a9cc75 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:12.473376 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb40ae-be12-41be-b0b8-c6e9194f542a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:12.474044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb40ae-be12-41be-b0b8-c6e9194f542a None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:12.474323 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb40ae-be12-41be-b0b8-c6e9194f542a None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:12.474597 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cb40ae-be12-41be-b0b8-c6e9194f542a None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:12.475436 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb40ae-be12-41be-b0b8-c6e9194f542a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:12.475755 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb40ae-be12-41be-b0b8-c6e9194f542a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:12.478473 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cb40ae-be12-41be-b0b8-c6e9194f542a None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:12.496192 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 618/1254] 10.222.0.61 () {56 vars in 1143 bytes} [Thu Jul 14 10:04:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:04:12.501851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416218b-6aad-40af-a0eb-96eab86f504f None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:12.608384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416218b-6aad-40af-a0eb-96eab86f504f None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:12.649334 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab16377-43be-441e-a868-ffd6fe944aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:12.650192 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab16377-43be-441e-a868-ffd6fe944aae None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:12.650666 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab16377-43be-441e-a868-ffd6fe944aae None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:12.650952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab16377-43be-441e-a868-ffd6fe944aae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:12.714337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:12.728797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab16377-43be-441e-a868-ffd6fe944aae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:12.730327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab16377-43be-441e-a868-ffd6fe944aae None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:12.838483 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 637/1255] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:04:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:04:12.844661 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb40ae-be12-41be-b0b8-c6e9194f542a None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:12.982851 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cb40ae-be12-41be-b0b8-c6e9194f542a None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:12.996939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:13.024106 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec73d4-6c0d-46d0-90e3-f600d7467884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:13.025130 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec73d4-6c0d-46d0-90e3-f600d7467884 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:13.025558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec73d4-6c0d-46d0-90e3-f600d7467884 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:13.025837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec73d4-6c0d-46d0-90e3-f600d7467884 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:13.039352 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec73d4-6c0d-46d0-90e3-f600d7467884 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:13.041133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec73d4-6c0d-46d0-90e3-f600d7467884 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:13.064668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-22ec73d4-6c0d-46d0-90e3-f600d7467884 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:13.064668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:13.064668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:13.064668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:13.064668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:13.064668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:13.064668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:13.064668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:13.064668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:13.064668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:13.064668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:13.064668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:13.064668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:04:13.064668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:04:13.064668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:13.065903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:13.072044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:04:13.072044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:04:13.072044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:13.072044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:13.072044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 619/1256] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:04:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:13.072414 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab16377-43be-441e-a868-ffd6fe944aae None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:13.166175 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:13.181431 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab16377-43be-441e-a868-ffd6fe944aae None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:13.220984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ad050-a2e1-4aff-863b-cdc7cd6bbf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:13.221772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ad050-a2e1-4aff-863b-cdc7cd6bbf54 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:13.222024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ad050-a2e1-4aff-863b-cdc7cd6bbf54 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:13.222346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00ad050-a2e1-4aff-863b-cdc7cd6bbf54 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:13.223078 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ad050-a2e1-4aff-863b-cdc7cd6bbf54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:13.223682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ad050-a2e1-4aff-863b-cdc7cd6bbf54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:13.225226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00ad050-a2e1-4aff-863b-cdc7cd6bbf54 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:13.238298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 638/1257] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:04:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:04:13.244803 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec73d4-6c0d-46d0-90e3-f600d7467884 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:13.351979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec73d4-6c0d-46d0-90e3-f600d7467884 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:13.387020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab3804-8663-4f21-bf5c-fb64b0b32f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:13.387799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab3804-8663-4f21-bf5c-fb64b0b32f79 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:13.388078 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab3804-8663-4f21-bf5c-fb64b0b32f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:13.388350 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab3804-8663-4f21-bf5c-fb64b0b32f79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:13.394886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab3804-8663-4f21-bf5c-fb64b0b32f79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:13.396327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab3804-8663-4f21-bf5c-fb64b0b32f79 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:13.410062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-76ab3804-8663-4f21-bf5c-fb64b0b32f79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:04:13.410062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:13.410062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:13.410062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:13.410062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:13.410062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:13.410062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:13.410062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:13.410062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:13.410062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:13.410062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:13.410062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:13.410062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:04:13.410062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:04:13.410062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:13.412114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:13.413416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:04:13.413416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:04:13.413416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:04:13.413416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:04:13.413416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:04:13.413416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:04:13.413416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:04:13.413416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:13.413416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 620/1258] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:04:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:13.418754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ad050-a2e1-4aff-863b-cdc7cd6bbf54 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:13.521909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00ad050-a2e1-4aff-863b-cdc7cd6bbf54 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:13.559272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a8186-475e-4d99-8ade-3115837f1624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:13.559935 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a8186-475e-4d99-8ade-3115837f1624 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:13.560213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a8186-475e-4d99-8ade-3115837f1624 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:13.560483 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a8186-475e-4d99-8ade-3115837f1624 None admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:13.561299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a8186-475e-4d99-8ade-3115837f1624 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:13.561603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a8186-475e-4d99-8ade-3115837f1624 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:13.563024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a8186-475e-4d99-8ade-3115837f1624 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:13.575763 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 639/1259] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:04:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:04:13.581929 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab3804-8663-4f21-bf5c-fb64b0b32f79 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:13.612026 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:13.696404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab3804-8663-4f21-bf5c-fb64b0b32f79 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:13.731711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd552fc1-bd67-407a-89a0-f6e970784b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:13.733415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd552fc1-bd67-407a-89a0-f6e970784b99 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:13.733415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd552fc1-bd67-407a-89a0-f6e970784b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:13.733415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd552fc1-bd67-407a-89a0-f6e970784b99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:13.733851 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd552fc1-bd67-407a-89a0-f6e970784b99 None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=57662256880f43018039715a755c087c, user.id=ff4c42a4f4f549378e5bc1cd92f801c7) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:13.734093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd552fc1-bd67-407a-89a0-f6e970784b99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:13.735508 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd552fc1-bd67-407a-89a0-f6e970784b99 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:13.741429 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 621/1260] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:04:13 2022] GET /identity/v3/role_assignments?user.id=ff4c42a4f4f549378e5bc1cd92f801c7&scope.project.id=57662256880f43018039715a755c087c&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:15.774081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607d114-baed-4230-a477-8db4995feb63 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:15.774603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607d114-baed-4230-a477-8db4995feb63 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:15.774603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607d114-baed-4230-a477-8db4995feb63 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:15.775969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 640/1261] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:04:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:04:15.781210 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e13b29-c0e6-4b5d-b93d-9a98142e5947 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:15.781593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e13b29-c0e6-4b5d-b93d-9a98142e5947 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:15.781768 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e13b29-c0e6-4b5d-b93d-9a98142e5947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:15.798455 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-91e13b29-c0e6-4b5d-b93d-9a98142e5947 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:04:15.799642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-91e13b29-c0e6-4b5d-b93d-9a98142e5947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:04:15.867996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 622/1262] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:04:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:04:15.890470 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607d114-baed-4230-a477-8db4995feb63 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:15.901874 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:15.990545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607d114-baed-4230-a477-8db4995feb63 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:16.025663 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69036455-5706-47fe-aee7-4d1f41535e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:16.026555 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69036455-5706-47fe-aee7-4d1f41535e8b None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:16.026833 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69036455-5706-47fe-aee7-4d1f41535e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:16.027167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69036455-5706-47fe-aee7-4d1f41535e8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:16.038394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69036455-5706-47fe-aee7-4d1f41535e8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-69036455-5706-47fe-aee7-4d1f41535e8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:04:16.046295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:16.047686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:16.048830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:16.048830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:16.048830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:16.048830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:16.048830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:04:16.048830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:04:16.048830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:04:16.048830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:04:16.048830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:04:16.048830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:04:16.048830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:04:16.048830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:04:16.048830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 641/1263] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:04:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:16.054663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e13b29-c0e6-4b5d-b93d-9a98142e5947 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:16.076185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:16.155714 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e13b29-c0e6-4b5d-b93d-9a98142e5947 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:16.188725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04d5b2-8e89-43c4-99df-288565917c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:16.189410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04d5b2-8e89-43c4-99df-288565917c4d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:16.189693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04d5b2-8e89-43c4-99df-288565917c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:16.189964 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be04d5b2-8e89-43c4-99df-288565917c4d None admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:16.190703 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04d5b2-8e89-43c4-99df-288565917c4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:16.191008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04d5b2-8e89-43c4-99df-288565917c4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:16.193022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be04d5b2-8e89-43c4-99df-288565917c4d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:16.200858 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 623/1264] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:04:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:16.206472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69036455-5706-47fe-aee7-4d1f41535e8b None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:16.270716 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:16.303673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69036455-5706-47fe-aee7-4d1f41535e8b None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:16.335696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca116b7-4c09-44ea-b5f3-0e6d70f8e692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:16.336472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca116b7-4c09-44ea-b5f3-0e6d70f8e692 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:16.336754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca116b7-4c09-44ea-b5f3-0e6d70f8e692 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:16.337057 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca116b7-4c09-44ea-b5f3-0e6d70f8e692 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:16.347328 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca116b7-4c09-44ea-b5f3-0e6d70f8e692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:16.349331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca116b7-4c09-44ea-b5f3-0e6d70f8e692 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:16.366398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-cca116b7-4c09-44ea-b5f3-0e6d70f8e692 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:16.366398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:16.366398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:16.366398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:16.366398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:16.366398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:16.366398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:16.366398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:16.366398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:16.366398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:16.366398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:16.366398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:16.366398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:04:16.366398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:04:16.366398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:16.367321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:16.368225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:04:16.368225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:04:16.368225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:16.368225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:04:16.368225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 642/1265] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:04:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:16.373353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04d5b2-8e89-43c4-99df-288565917c4d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:16.451938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:16.472121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be04d5b2-8e89-43c4-99df-288565917c4d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:16.505036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71d813-07a5-4bc6-9fba-2f040596494c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:16.505687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71d813-07a5-4bc6-9fba-2f040596494c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:16.505973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71d813-07a5-4bc6-9fba-2f040596494c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:16.506245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71d813-07a5-4bc6-9fba-2f040596494c None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:16.506964 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71d813-07a5-4bc6-9fba-2f040596494c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:16.507268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71d813-07a5-4bc6-9fba-2f040596494c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:16.508620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af71d813-07a5-4bc6-9fba-2f040596494c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:16.519978 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 624/1266] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:16.525879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca116b7-4c09-44ea-b5f3-0e6d70f8e692 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:16.623961 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca116b7-4c09-44ea-b5f3-0e6d70f8e692 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:16.642754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:16.656455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab5d4e-36cb-4135-8209-8f458f0f708c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:16.657277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab5d4e-36cb-4135-8209-8f458f0f708c None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:16.657557 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab5d4e-36cb-4135-8209-8f458f0f708c None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:16.657829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab5d4e-36cb-4135-8209-8f458f0f708c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:16.665524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab5d4e-36cb-4135-8209-8f458f0f708c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:16.666944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab5d4e-36cb-4135-8209-8f458f0f708c None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-07ab5d4e-36cb-4135-8209-8f458f0f708c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:04:16.681659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:16.682778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:16.684178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:16.684178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:16.684178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:04:16.684178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:04:16.684178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:04:16.684178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:04:16.684178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:04:16.684178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:04:16.684178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:04:16.684178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:04:16.684178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 643/1267] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:04:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:16.688635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71d813-07a5-4bc6-9fba-2f040596494c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:16.785238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71d813-07a5-4bc6-9fba-2f040596494c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:16.818074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933651ba-6fc4-4b88-8ef8-babc5a8677cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:16.818725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933651ba-6fc4-4b88-8ef8-babc5a8677cd None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:16.819005 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933651ba-6fc4-4b88-8ef8-babc5a8677cd None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:16.819279 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933651ba-6fc4-4b88-8ef8-babc5a8677cd None admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:16.820107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933651ba-6fc4-4b88-8ef8-babc5a8677cd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:16.820420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933651ba-6fc4-4b88-8ef8-babc5a8677cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:16.821850 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933651ba-6fc4-4b88-8ef8-babc5a8677cd None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:16.830147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:16.839606 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 625/1268] 10.222.0.61 () {56 vars in 1143 bytes} [Thu Jul 14 10:04:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:16.845091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab5d4e-36cb-4135-8209-8f458f0f708c None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:16.941707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab5d4e-36cb-4135-8209-8f458f0f708c None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:16.973986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50372468-5a09-4a68-a830-12aa56f9d4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:16.974757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50372468-5a09-4a68-a830-12aa56f9d4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:16.975030 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50372468-5a09-4a68-a830-12aa56f9d4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:16.975298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50372468-5a09-4a68-a830-12aa56f9d4b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:16.985362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50372468-5a09-4a68-a830-12aa56f9d4b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:16.986792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50372468-5a09-4a68-a830-12aa56f9d4b4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:17.003277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-50372468-5a09-4a68-a830-12aa56f9d4b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:17.003277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:17.003277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:17.003277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:17.003277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:17.003277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:17.003277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:17.003277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:17.003277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:17.003277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:17.003277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:17.003277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:17.003277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:04:17.003277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:04:17.003277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:17.004213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:17.005390 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:04:17.005390 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:04:17.005390 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:17.005390 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:04:17.005390 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 644/1269] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:04:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:17.010892 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933651ba-6fc4-4b88-8ef8-babc5a8677cd None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:17.116191 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933651ba-6fc4-4b88-8ef8-babc5a8677cd None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:17.149918 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1c3b1-fae8-4ac1-ba41-0504cf9b990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:17.150576 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1c3b1-fae8-4ac1-ba41-0504cf9b990c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:17.150857 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1c3b1-fae8-4ac1-ba41-0504cf9b990c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:17.151132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d1c3b1-fae8-4ac1-ba41-0504cf9b990c None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:17.151853 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1c3b1-fae8-4ac1-ba41-0504cf9b990c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:17.152161 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1c3b1-fae8-4ac1-ba41-0504cf9b990c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:17.153552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d1c3b1-fae8-4ac1-ba41-0504cf9b990c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:17.166504 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 626/1270] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:04:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:04:17.172126 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50372468-5a09-4a68-a830-12aa56f9d4b4 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:17.203778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:17.271622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50372468-5a09-4a68-a830-12aa56f9d4b4 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:17.303433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ba735-3a68-4ffc-9ea4-bb9afb40d275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:17.304217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ba735-3a68-4ffc-9ea4-bb9afb40d275 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:17.304499 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ba735-3a68-4ffc-9ea4-bb9afb40d275 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:17.304771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388ba735-3a68-4ffc-9ea4-bb9afb40d275 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:17.311251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ba735-3a68-4ffc-9ea4-bb9afb40d275 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:17.312676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388ba735-3a68-4ffc-9ea4-bb9afb40d275 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:17.327172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-388ba735-3a68-4ffc-9ea4-bb9afb40d275 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:04:17.327172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:17.327172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:17.327172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:17.327172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:17.327172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:17.327172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:17.327172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:17.327172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:17.327172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:17.327172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:17.327172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:17.327172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:04:17.327172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:04:17.327172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:17.328211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:17.329267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:04:17.329267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:04:17.329267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:04:17.329267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:04:17.329267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:04:17.329267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:04:17.329267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:04:17.329267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:04:17.329267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 645/1271] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:04:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:17.334770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1c3b1-fae8-4ac1-ba41-0504cf9b990c None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:17.391454 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:17.433292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d1c3b1-fae8-4ac1-ba41-0504cf9b990c None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:17.466377 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa969d0b-683f-453b-84fe-f8b995aab598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:17.466939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa969d0b-683f-453b-84fe-f8b995aab598 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:17.467216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa969d0b-683f-453b-84fe-f8b995aab598 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:17.467488 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa969d0b-683f-453b-84fe-f8b995aab598 None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:17.468348 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa969d0b-683f-453b-84fe-f8b995aab598 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:17.468746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa969d0b-683f-453b-84fe-f8b995aab598 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:17.470398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa969d0b-683f-453b-84fe-f8b995aab598 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:17.482686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 627/1272] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:04:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:17.488560 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ba735-3a68-4ffc-9ea4-bb9afb40d275 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:17.575925 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:17.584611 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388ba735-3a68-4ffc-9ea4-bb9afb40d275 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:17.617510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c801a43-0e5b-4a5a-abbb-4a536224a1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:17.618391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c801a43-0e5b-4a5a-abbb-4a536224a1c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:17.618672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c801a43-0e5b-4a5a-abbb-4a536224a1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:17.618945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c801a43-0e5b-4a5a-abbb-4a536224a1c0 None admin] PATH_INFO: `/v3/projects/57662256880f43018039715a755c087c/users/ff4c42a4f4f549378e5bc1cd92f801c7/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:17.620872 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c801a43-0e5b-4a5a-abbb-4a536224a1c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=57662256880f43018039715a755c087c, user_id=ff4c42a4f4f549378e5bc1cd92f801c7, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:17.623287 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c801a43-0e5b-4a5a-abbb-4a536224a1c0 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:17.636215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 646/1273] 10.222.0.61 () {58 vars in 1439 bytes} [Thu Jul 14 10:04:17 2022] PUT /identity/v3/projects/57662256880f43018039715a755c087c/users/ff4c42a4f4f549378e5bc1cd92f801c7/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:04:19.616830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15854032-04ff-4dd9-8846-00b4247c780f None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:19.617502 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15854032-04ff-4dd9-8846-00b4247c780f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:19.617502 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15854032-04ff-4dd9-8846-00b4247c780f None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:19.618694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 628/1274] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:04:19.623683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4e6ea-73fb-442a-8f62-2dfd032e1044 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:19.624053 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4e6ea-73fb-442a-8f62-2dfd032e1044 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:19.624227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4e6ea-73fb-442a-8f62-2dfd032e1044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:19.640470 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5bc4e6ea-73fb-442a-8f62-2dfd032e1044 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:04:19.641665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5bc4e6ea-73fb-442a-8f62-2dfd032e1044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:04:19.706085 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 647/1275] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:04:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:04:19.730284 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15854032-04ff-4dd9-8846-00b4247c780f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:19.828545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15854032-04ff-4dd9-8846-00b4247c780f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:19.845633 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:19.863061 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb6235-bbf3-4369-a309-d49cd7a889e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:19.863936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb6235-bbf3-4369-a309-d49cd7a889e5 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:19.864219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb6235-bbf3-4369-a309-d49cd7a889e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:19.864495 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb6235-bbf3-4369-a309-d49cd7a889e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:19.874421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb6235-bbf3-4369-a309-d49cd7a889e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-c5eb6235-bbf3-4369-a309-d49cd7a889e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 14 10:04:19.882101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:19.883196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:19.884256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:19.884256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:19.884256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:19.884256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:19.884256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 14 10:04:19.884256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 14 10:04:19.884256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 14 10:04:19.884256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 14 10:04:19.884256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 14 10:04:19.884256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 14 10:04:19.884256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 14 10:04:19.884256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:19.884256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 629/1276] 10.222.0.61 () {56 vars in 1096 bytes} [Thu Jul 14 10:04:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:19.889243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4e6ea-73fb-442a-8f62-2dfd032e1044 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:19.966666 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:19.985771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4e6ea-73fb-442a-8f62-2dfd032e1044 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:20.022792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68a839-779a-4fd4-a6d7-5bee212eeb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:20.023970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68a839-779a-4fd4-a6d7-5bee212eeb02 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:20.024377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68a839-779a-4fd4-a6d7-5bee212eeb02 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:20.024807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68a839-779a-4fd4-a6d7-5bee212eeb02 None admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:20.026072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68a839-779a-4fd4-a6d7-5bee212eeb02 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:20.026525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68a839-779a-4fd4-a6d7-5bee212eeb02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:20.028670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68a839-779a-4fd4-a6d7-5bee212eeb02 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:20.038268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 648/1277] 10.222.0.61 () {56 vars in 1099 bytes} [Thu Jul 14 10:04:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:20.044114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb6235-bbf3-4369-a309-d49cd7a889e5 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:20.141676 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb6235-bbf3-4369-a309-d49cd7a889e5 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:20.174319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0392e-071b-4579-a2ed-60cc3d03bf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:20.175091 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0392e-071b-4579-a2ed-60cc3d03bf5f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:20.175369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0392e-071b-4579-a2ed-60cc3d03bf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:20.175640 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0392e-071b-4579-a2ed-60cc3d03bf5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:20.185787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0392e-071b-4579-a2ed-60cc3d03bf5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:20.187206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e0392e-071b-4579-a2ed-60cc3d03bf5f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:20.204360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-69e0392e-071b-4579-a2ed-60cc3d03bf5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:20.204360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:20.204360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:20.204360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:20.204360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:20.204360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:20.204360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:20.204360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:20.204360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:20.204360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:20.204360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:20.204360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:20.204360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:04:20.204360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:04:20.204360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:20.205418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:20.206621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:04:20.206621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:04:20.206621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:20.206621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:20.206621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 630/1278] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:04:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:20.211623 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68a839-779a-4fd4-a6d7-5bee212eeb02 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:20.306577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68a839-779a-4fd4-a6d7-5bee212eeb02 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:20.338376 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac8bd6-5a48-4630-9a90-ca3016f66ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:20.339052 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac8bd6-5a48-4630-9a90-ca3016f66ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:20.339351 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac8bd6-5a48-4630-9a90-ca3016f66ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:20.339620 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac8bd6-5a48-4630-9a90-ca3016f66ef1 None admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:20.340331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac8bd6-5a48-4630-9a90-ca3016f66ef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:20.340632 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac8bd6-5a48-4630-9a90-ca3016f66ef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:20.342028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bac8bd6-5a48-4630-9a90-ca3016f66ef1 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:20.353261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 649/1279] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:20.358844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0392e-071b-4579-a2ed-60cc3d03bf5f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:20.388829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:20.455988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0392e-071b-4579-a2ed-60cc3d03bf5f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:20.487543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d5385-a387-48a5-9ad7-1fd5164d8a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:20.488313 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d5385-a387-48a5-9ad7-1fd5164d8a4b None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:20.488589 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d5385-a387-48a5-9ad7-1fd5164d8a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:20.489423 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d5385-a387-48a5-9ad7-1fd5164d8a4b None admin] PATH_INFO: `/v3/users/placement` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:20.496559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d5385-a387-48a5-9ad7-1fd5164d8a4b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:20.498000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779d5385-a387-48a5-9ad7-1fd5164d8a4b None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-779d5385-a387-48a5-9ad7-1fd5164d8a4b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 14 10:04:20.512255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:20.513397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:20.514658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:20.514658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:20.514658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 14 10:04:20.514658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 14 10:04:20.514658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 14 10:04:20.514658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 14 10:04:20.514658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 14 10:04:20.514658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 14 10:04:20.514658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 14 10:04:20.514658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:20.514658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 631/1280] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:04:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:20.518972 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac8bd6-5a48-4630-9a90-ca3016f66ef1 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:20.522622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:20.618335 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac8bd6-5a48-4630-9a90-ca3016f66ef1 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:20.651003 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dca6f-08c0-4ce6-be74-212c3b108689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:20.651660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dca6f-08c0-4ce6-be74-212c3b108689 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:20.651942 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dca6f-08c0-4ce6-be74-212c3b108689 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:20.652214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5dca6f-08c0-4ce6-be74-212c3b108689 None admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:20.653080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5dca6f-08c0-4ce6-be74-212c3b108689 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:20.653390 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5dca6f-08c0-4ce6-be74-212c3b108689 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:20.654800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5dca6f-08c0-4ce6-be74-212c3b108689 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:20.671255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 650/1281] 10.222.0.61 () {56 vars in 1143 bytes} [Thu Jul 14 10:04:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:20.677622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d5385-a387-48a5-9ad7-1fd5164d8a4b None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:20.766642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:20.776268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779d5385-a387-48a5-9ad7-1fd5164d8a4b None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:20.809233 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0551de1-705a-46ef-b268-fc6d87d73084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:20.809870 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0551de1-705a-46ef-b268-fc6d87d73084 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:20.810153 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0551de1-705a-46ef-b268-fc6d87d73084 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:20.810429 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0551de1-705a-46ef-b268-fc6d87d73084 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:20.875593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0551de1-705a-46ef-b268-fc6d87d73084 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:20.877055 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0551de1-705a-46ef-b268-fc6d87d73084 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:20.962606 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:20.972600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 632/1282] 10.222.0.61 () {58 vars in 1282 bytes} [Thu Jul 14 10:04:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 14 10:04:20.978804 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dca6f-08c0-4ce6-be74-212c3b108689 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:21.039398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:21.076687 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5dca6f-08c0-4ce6-be74-212c3b108689 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:21.109129 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75233cf3-3e1a-4e07-9b80-c272a815df62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:21.109907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75233cf3-3e1a-4e07-9b80-c272a815df62 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:21.110188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75233cf3-3e1a-4e07-9b80-c272a815df62 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:21.110462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75233cf3-3e1a-4e07-9b80-c272a815df62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:21.120947 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75233cf3-3e1a-4e07-9b80-c272a815df62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:21.122384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75233cf3-3e1a-4e07-9b80-c272a815df62 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:21.139365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-75233cf3-3e1a-4e07-9b80-c272a815df62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:21.139365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:21.139365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:21.139365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:21.139365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:21.139365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:21.139365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:21.139365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:21.139365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:21.139365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:21.139365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:21.139365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:21.139365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 14 10:04:21.139365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 14 10:04:21.139365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:21.140388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:21.141391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 14 10:04:21.141391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 14 10:04:21.141391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 14 10:04:21.141391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:04:21.141391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 651/1283] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:04:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:21.146217 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0551de1-705a-46ef-b268-fc6d87d73084 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:21.243602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0551de1-705a-46ef-b268-fc6d87d73084 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:21.277773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f1d8c-3cfa-476e-8467-53a6571a2f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:21.278425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f1d8c-3cfa-476e-8467-53a6571a2f2d None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:21.278704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f1d8c-3cfa-476e-8467-53a6571a2f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:21.278974 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49f1d8c-3cfa-476e-8467-53a6571a2f2d None admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:21.279702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f1d8c-3cfa-476e-8467-53a6571a2f2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:21.280004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f1d8c-3cfa-476e-8467-53a6571a2f2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:21.281389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49f1d8c-3cfa-476e-8467-53a6571a2f2d None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:21.293299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 633/1284] 10.222.0.61 () {56 vars in 1109 bytes} [Thu Jul 14 10:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:21.298830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75233cf3-3e1a-4e07-9b80-c272a815df62 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:21.394092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75233cf3-3e1a-4e07-9b80-c272a815df62 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:21.408026 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:21.428667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9e211-46d9-4864-bde8-792bfbe260bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:21.429486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9e211-46d9-4864-bde8-792bfbe260bc None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:21.429770 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9e211-46d9-4864-bde8-792bfbe260bc None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:21.430122 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9e211-46d9-4864-bde8-792bfbe260bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:21.436583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9e211-46d9-4864-bde8-792bfbe260bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:21.438049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9e211-46d9-4864-bde8-792bfbe260bc None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:21.451564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application [None req-06d9e211-46d9-4864-bde8-792bfbe260bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:04:21.451564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:21.451564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:21.451564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:21.451564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:21.451564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:21.451564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:21.451564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:21.451564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:21.451564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:21.451564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:21.451564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:21.451564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 14 10:04:21.451564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 14 10:04:21.451564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:21.452583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:21.453826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 14 10:04:21.453826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 14 10:04:21.453826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 14 10:04:21.453826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 14 10:04:21.453826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 14 10:04:21.453826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 14 10:04:21.453826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 14 10:04:21.453826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: ERROR keystone.server.flask.application Jul 14 10:04:21.453826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 652/1285] 10.222.0.61 () {56 vars in 1111 bytes} [Thu Jul 14 10:04:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:21.458988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f1d8c-3cfa-476e-8467-53a6571a2f2d None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:21.507041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:21.557957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49f1d8c-3cfa-476e-8467-53a6571a2f2d None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:21.590262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebbe4d-c299-44ed-8485-768aa47ca70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:21.590901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebbe4d-c299-44ed-8485-768aa47ca70c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:21.591177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebbe4d-c299-44ed-8485-768aa47ca70c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:21.591446 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebbe4d-c299-44ed-8485-768aa47ca70c None admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:21.592222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebbe4d-c299-44ed-8485-768aa47ca70c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:21.592538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebbe4d-c299-44ed-8485-768aa47ca70c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:21.593968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebbe4d-c299-44ed-8485-768aa47ca70c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:21.605431 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 634/1286] 10.222.0.61 () {56 vars in 1148 bytes} [Thu Jul 14 10:04:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:21.611521 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9e211-46d9-4864-bde8-792bfbe260bc None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:21.707042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9e211-46d9-4864-bde8-792bfbe260bc None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:21.739018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0786f-327d-496a-ab7e-66712e0c1986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:21.739649 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0786f-327d-496a-ab7e-66712e0c1986 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:21.739927 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0786f-327d-496a-ab7e-66712e0c1986 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:21.740197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0786f-327d-496a-ab7e-66712e0c1986 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:21.741060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0786f-327d-496a-ab7e-66712e0c1986 None admin] RBAC: Adding query filter params (role.id=3b5aa52426c743e7904ebec79cd537cb, scope.project.id=57662256880f43018039715a755c087c, user.id=ff4c42a4f4f549378e5bc1cd92f801c7) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:21.741367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0786f-327d-496a-ab7e-66712e0c1986 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:21.742788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0786f-327d-496a-ab7e-66712e0c1986 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:21.748320 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 653/1287] 10.222.0.61 () {56 vars in 1374 bytes} [Thu Jul 14 10:04:21 2022] GET /identity/v3/role_assignments?user.id=ff4c42a4f4f549378e5bc1cd92f801c7&scope.project.id=57662256880f43018039715a755c087c&role.id=3b5aa52426c743e7904ebec79cd537cb => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:23.826586 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f92b1-467d-4a32-adc7-c000879a7882 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:23.827133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f92b1-467d-4a32-adc7-c000879a7882 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:23.827133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f92b1-467d-4a32-adc7-c000879a7882 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:23.828498 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 635/1288] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:04:23.833615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c5962-dab9-44fa-b87c-0be7bd5ab444 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:23.834031 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c5962-dab9-44fa-b87c-0be7bd5ab444 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:23.834174 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c5962-dab9-44fa-b87c-0be7bd5ab444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:23.851053 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-078c5962-dab9-44fa-b87c-0be7bd5ab444 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:04:23.852229 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-078c5962-dab9-44fa-b87c-0be7bd5ab444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:04:23.880796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:23.917145 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 654/1289] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:04:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:04:23.941213 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f92b1-467d-4a32-adc7-c000879a7882 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:24.049731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f92b1-467d-4a32-adc7-c000879a7882 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:24.064407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:24.092175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e2c00-7eae-4273-80a6-09f3861e1b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:24.093050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e2c00-7eae-4273-80a6-09f3861e1b0f None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:24.093470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e2c00-7eae-4273-80a6-09f3861e1b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:24.093754 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3e2c00-7eae-4273-80a6-09f3861e1b0f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:24.094412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e2c00-7eae-4273-80a6-09f3861e1b0f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:24.095774 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3e2c00-7eae-4273-80a6-09f3861e1b0f None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-4b3e2c00-7eae-4273-80a6-09f3861e1b0f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:24.107872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:24.109148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:24.110043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:24.110043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:24.110043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:24.110043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:24.110043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:24.110043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:24.110043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:24.110043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:24.110043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:24.110043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:24.110043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:04:24.110043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:04:24.110043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:04:24.110043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:24.110043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 636/1290] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Jul 14 10:04:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:24.115705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c5962-dab9-44fa-b87c-0be7bd5ab444 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:24.211337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c5962-dab9-44fa-b87c-0be7bd5ab444 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:24.243976 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17daa7e5-a5ad-442f-a775-05de7003e51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:24.244644 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daa7e5-a5ad-442f-a775-05de7003e51f None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:24.244945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daa7e5-a5ad-442f-a775-05de7003e51f None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:24.245225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17daa7e5-a5ad-442f-a775-05de7003e51f None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:24.245935 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17daa7e5-a5ad-442f-a775-05de7003e51f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:24.246242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17daa7e5-a5ad-442f-a775-05de7003e51f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:24.247598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17daa7e5-a5ad-442f-a775-05de7003e51f None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:24.252390 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 655/1291] 10.222.0.61 () {56 vars in 1116 bytes} [Thu Jul 14 10:04:24 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:24.258047 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e2c00-7eae-4273-80a6-09f3861e1b0f None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:24.355446 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3e2c00-7eae-4273-80a6-09f3861e1b0f None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:24.386927 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4474bba-f245-4c7a-8bab-14665ea6f32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:24.387572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4474bba-f245-4c7a-8bab-14665ea6f32c None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:24.387852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4474bba-f245-4c7a-8bab-14665ea6f32c None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:24.388125 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4474bba-f245-4c7a-8bab-14665ea6f32c None admin] PATH_INFO: `/v3/services` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:24.388824 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4474bba-f245-4c7a-8bab-14665ea6f32c None admin] RBAC: Adding query filter params (type=placement) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:24.389163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4474bba-f245-4c7a-8bab-14665ea6f32c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:24.390513 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4474bba-f245-4c7a-8bab-14665ea6f32c None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:24.395361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 637/1292] 10.222.0.61 () {56 vars in 1116 bytes} [Thu Jul 14 10:04:24 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:26.386091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0de52-9c26-47df-865a-641849dd88e4 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:26.386712 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0de52-9c26-47df-865a-641849dd88e4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:26.386712 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e0de52-9c26-47df-865a-641849dd88e4 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:26.387992 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 656/1293] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:04:26.393258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a7bfe-209c-4d4b-8f1a-86b073e87849 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:26.393678 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a7bfe-209c-4d4b-8f1a-86b073e87849 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:26.393823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0a7bfe-209c-4d4b-8f1a-86b073e87849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:26.411144 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-db0a7bfe-209c-4d4b-8f1a-86b073e87849 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:04:26.412321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-db0a7bfe-209c-4d4b-8f1a-86b073e87849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:04:26.445685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:26.481531 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 638/1294] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:04:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:04:26.504077 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0de52-9c26-47df-865a-641849dd88e4 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:26.536475 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:26.605178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e0de52-9c26-47df-865a-641849dd88e4 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:26.637043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97448680-5275-46cc-a5b0-d0c52d4c6fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:26.637699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97448680-5275-46cc-a5b0-d0c52d4c6fe4 None admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:26.637982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97448680-5275-46cc-a5b0-d0c52d4c6fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:26.638258 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97448680-5275-46cc-a5b0-d0c52d4c6fe4 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:26.638993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97448680-5275-46cc-a5b0-d0c52d4c6fe4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:26.640344 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97448680-5275-46cc-a5b0-d0c52d4c6fe4 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:26.653801 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 657/1295] 10.222.0.61 () {60 vars in 1141 bytes} [Thu Jul 14 10:04:26 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:04:28.661244 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cd575-821e-4623-912f-d519ae7fb960 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:28.661773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cd575-821e-4623-912f-d519ae7fb960 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:28.661773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493cd575-821e-4623-912f-d519ae7fb960 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:28.663148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 639/1296] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:04:28.668422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ff516-038f-4d96-8e06-e02b9daf80a3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:28.668865 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ff516-038f-4d96-8e06-e02b9daf80a3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:28.668990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777ff516-038f-4d96-8e06-e02b9daf80a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:28.685967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-777ff516-038f-4d96-8e06-e02b9daf80a3 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:04:28.687148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-777ff516-038f-4d96-8e06-e02b9daf80a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:04:28.756844 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 658/1297] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:04:28 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:04:28.778532 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cd575-821e-4623-912f-d519ae7fb960 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:28.875213 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493cd575-821e-4623-912f-d519ae7fb960 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:28.908229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2301c-483c-4fa2-89cb-ff71d48e7901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:28.909028 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2301c-483c-4fa2-89cb-ff71d48e7901 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:28.909309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2301c-483c-4fa2-89cb-ff71d48e7901 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:28.909611 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd2301c-483c-4fa2-89cb-ff71d48e7901 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:28.910276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2301c-483c-4fa2-89cb-ff71d48e7901 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:28.911696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd2301c-483c-4fa2-89cb-ff71d48e7901 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-7fd2301c-483c-4fa2-89cb-ff71d48e7901 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:28.924229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:28.925547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:28.927412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:28.927412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:28.927412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:28.927412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:28.927412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:28.927412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:28.927412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:28.927412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:28.927412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:28.927412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:28.927412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:04:28.927412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:04:28.927412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:04:28.927412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:28.927412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 640/1298] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Jul 14 10:04:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:28.931953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ff516-038f-4d96-8e06-e02b9daf80a3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:28.950463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:29.040772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777ff516-038f-4d96-8e06-e02b9daf80a3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:29.079564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bd425-0a57-4483-a2c0-ce1c9b58a2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:29.080346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bd425-0a57-4483-a2c0-ce1c9b58a2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:29.080689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bd425-0a57-4483-a2c0-ce1c9b58a2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:29.081007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bd425-0a57-4483-a2c0-ce1c9b58a2b9 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:29.081830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bd425-0a57-4483-a2c0-ce1c9b58a2b9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:29.082160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bd425-0a57-4483-a2c0-ce1c9b58a2b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:29.084278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bd425-0a57-4483-a2c0-ce1c9b58a2b9 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:29.089394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 659/1299] 10.222.0.61 () {56 vars in 1116 bytes} [Thu Jul 14 10:04:28 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:29.095357 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2301c-483c-4fa2-89cb-ff71d48e7901 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:29.126722 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:29.194294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd2301c-483c-4fa2-89cb-ff71d48e7901 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:29.226006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dd16e8-6b36-44da-97a0-684fa602d103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:29.226666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd16e8-6b36-44da-97a0-684fa602d103 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:29.226940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd16e8-6b36-44da-97a0-684fa602d103 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:29.227206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dd16e8-6b36-44da-97a0-684fa602d103 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:29.227984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd16e8-6b36-44da-97a0-684fa602d103 None admin] RBAC: Adding query filter params (interface=public, service_id=c3c9e23f788946d3806945bf039542b4, region_id=RegionOne) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:29.228287 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd16e8-6b36-44da-97a0-684fa602d103 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:29.229648 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dd16e8-6b36-44da-97a0-684fa602d103 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:29.234446 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 641/1300] 10.222.0.61 () {56 vars in 1251 bytes} [Thu Jul 14 10:04:29 2022] GET /identity/v3/endpoints?service_id=c3c9e23f788946d3806945bf039542b4&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:04:29.240110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bd425-0a57-4483-a2c0-ce1c9b58a2b9 None admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:29.325904 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:29.344081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bd425-0a57-4483-a2c0-ce1c9b58a2b9 None admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:29.375990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e0e5b-2cfe-4599-a59f-8dd2cedd9ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:29.376639 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e0e5b-2cfe-4599-a59f-8dd2cedd9ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:29.376937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e0e5b-2cfe-4599-a59f-8dd2cedd9ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:29.377225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e0e5b-2cfe-4599-a59f-8dd2cedd9ca2 None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:29.377904 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e0e5b-2cfe-4599-a59f-8dd2cedd9ca2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:29.379783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e0e5b-2cfe-4599-a59f-8dd2cedd9ca2 None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:29.386364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 660/1301] 10.222.0.61 () {56 vars in 1087 bytes} [Thu Jul 14 10:04:29 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:04:31.413281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a4943-2d62-4c83-985f-454e84509eee None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:31.413832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a4943-2d62-4c83-985f-454e84509eee None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:31.413832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a4943-2d62-4c83-985f-454e84509eee None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:31.415219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 642/1302] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:04:31.420505 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44271e5f-2c8c-4c24-a655-1c6746fcfd30 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:31.420964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44271e5f-2c8c-4c24-a655-1c6746fcfd30 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:31.421072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44271e5f-2c8c-4c24-a655-1c6746fcfd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:31.437984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-44271e5f-2c8c-4c24-a655-1c6746fcfd30 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:04:31.439160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-44271e5f-2c8c-4c24-a655-1c6746fcfd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:04:31.502949 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 661/1303] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:04:31 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 14 10:04:31.525015 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a4943-2d62-4c83-985f-454e84509eee None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:31.617927 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:31.624791 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a4943-2d62-4c83-985f-454e84509eee None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:31.656794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508346c5-fa2b-494b-a1ab-87e06f4940a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:31.657602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508346c5-fa2b-494b-a1ab-87e06f4940a3 None admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:31.657880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508346c5-fa2b-494b-a1ab-87e06f4940a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:31.658149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508346c5-fa2b-494b-a1ab-87e06f4940a3 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:31.658795 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508346c5-fa2b-494b-a1ab-87e06f4940a3 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:31.660146 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508346c5-fa2b-494b-a1ab-87e06f4940a3 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application [None req-508346c5-fa2b-494b-a1ab-87e06f4940a3 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 14 10:04:31.672430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 14 10:04:31.673460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.get_or_create( Jul 14 10:04:31.674383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 14 10:04:31.674383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application with Lock( Jul 14 10:04:31.674383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 14 10:04:31.674383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self._enter() Jul 14 10:04:31.674383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 14 10:04:31.674383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 14 10:04:31.674383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 14 10:04:31.674383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application return self.creator() Jul 14 10:04:31.674383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 14 10:04:31.674383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application created_value = creator( Jul 14 10:04:31.674383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 14 10:04:31.674383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 14 10:04:31.674383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 14 10:04:31.674383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: ERROR keystone.server.flask.application Jul 14 10:04:31.674383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 643/1304] 10.222.0.61 () {56 vars in 1117 bytes} [Thu Jul 14 10:04:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 14 10:04:31.680238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44271e5f-2c8c-4c24-a655-1c6746fcfd30 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:31.755244 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:31.777748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44271e5f-2c8c-4c24-a655-1c6746fcfd30 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:31.810510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e37c0-d452-4e00-a0a8-1f534fce4b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:31.811163 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e37c0-d452-4e00-a0a8-1f534fce4b8d None admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:31.811444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e37c0-d452-4e00-a0a8-1f534fce4b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:31.811717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e37c0-d452-4e00-a0a8-1f534fce4b8d None admin] PATH_INFO: `/v3/services` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:31.812420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e37c0-d452-4e00-a0a8-1f534fce4b8d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:04:31.812727 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e37c0-d452-4e00-a0a8-1f534fce4b8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:31.814115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3e37c0-d452-4e00-a0a8-1f534fce4b8d None admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:31.819138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 662/1305] 10.222.0.61 () {56 vars in 1116 bytes} [Thu Jul 14 10:04:31 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:04:31.824907 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508346c5-fa2b-494b-a1ab-87e06f4940a3 None admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:04:31.921295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508346c5-fa2b-494b-a1ab-87e06f4940a3 None admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:04:31.952625 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92130313-1b66-4b27-b8a4-d89a62e4c501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:04:31.953290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92130313-1b66-4b27-b8a4-d89a62e4c501 None admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:04:31.953571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92130313-1b66-4b27-b8a4-d89a62e4c501 None admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:04:31.953843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92130313-1b66-4b27-b8a4-d89a62e4c501 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:04:31.954582 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92130313-1b66-4b27-b8a4-d89a62e4c501 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:04:31.955924 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92130313-1b66-4b27-b8a4-d89a62e4c501 None admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:04:31.974354 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 644/1306] 10.222.0.61 () {60 vars in 1144 bytes} [Thu Jul 14 10:04:31 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:04:41.816691 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=30 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:04:41.969654 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:34.380448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0ab2e-341b-4952-9e1d-ddaf1190d8ea None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:34.381160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0ab2e-341b-4952-9e1d-ddaf1190d8ea None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:34.381160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba0ab2e-341b-4952-9e1d-ddaf1190d8ea None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:34.387397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b8c37-e553-475d-9925-29e6df43268c None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:34.387832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b8c37-e553-475d-9925-29e6df43268c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:34.387927 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b8c37-e553-475d-9925-29e6df43268c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:34.391393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 663/1307] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:05:34 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:05:34.461885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-238b8c37-e553-475d-9925-29e6df43268c None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:05:34.463103 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-238b8c37-e553-475d-9925-29e6df43268c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:34.577070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 645/1308] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:05:34.639424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3434dd5-8896-4d3a-894f-5655b96febea None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:34.640032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3434dd5-8896-4d3a-894f-5655b96febea None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:34.640615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3434dd5-8896-4d3a-894f-5655b96febea None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:34.643401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 664/1309] 10.222.0.61 () {54 vars in 964 bytes} [Thu Jul 14 10:05:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:05:34.651331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3950ac-939a-4011-bacd-e82df08eea35 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:34.651715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3950ac-939a-4011-bacd-e82df08eea35 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:34.652188 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3950ac-939a-4011-bacd-e82df08eea35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:34.756135 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-1f3950ac-939a-4011-bacd-e82df08eea35 None None] MFA Rules not processed for user `cd6d19c4406b4b228065b98a9a074b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:05:34.758016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1f3950ac-939a-4011-bacd-e82df08eea35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:34.788315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:34.874889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3434dd5-8896-4d3a-894f-5655b96febea None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:05:34.877188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e3434dd5-8896-4d3a-894f-5655b96febea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:34.882523 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 646/1310] 10.222.0.61 () {58 vars in 1061 bytes} [Thu Jul 14 10:05:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:05:35.022309 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3434dd5-8896-4d3a-894f-5655b96febea None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:05:35.103803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc162e59-3e6b-4747-8de1-538fcb2703be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:05:35.104474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc162e59-3e6b-4747-8de1-538fcb2703be service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:35.104752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc162e59-3e6b-4747-8de1-538fcb2703be service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:35.105051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc162e59-3e6b-4747-8de1-538fcb2703be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:35.106708 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bc162e59-3e6b-4747-8de1-538fcb2703be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:35.164677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:35.220378 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc162e59-3e6b-4747-8de1-538fcb2703be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:05:35.222436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc162e59-3e6b-4747-8de1-538fcb2703be service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:05:35.243159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 665/1311] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:05:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:05:37.579250 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4e037-6e61-4aec-9c98-9247393f71c4 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:37.579766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4e037-6e61-4aec-9c98-9247393f71c4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:37.579766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d4e037-6e61-4aec-9c98-9247393f71c4 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:37.581331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 647/1312] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:05:37.586398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa456f5e-dbb9-461f-9690-06b50fe0b7ad None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:37.586812 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa456f5e-dbb9-461f-9690-06b50fe0b7ad None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:37.586953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa456f5e-dbb9-461f-9690-06b50fe0b7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:37.678464 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-aa456f5e-dbb9-461f-9690-06b50fe0b7ad None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:05:37.679892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-aa456f5e-dbb9-461f-9690-06b50fe0b7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:37.799205 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 666/1313] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:05:37.837468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4e037-6e61-4aec-9c98-9247393f71c4 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:05:37.839007 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-80d4e037-6e61-4aec-9c98-9247393f71c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:37.949732 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d4e037-6e61-4aec-9c98-9247393f71c4 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:05:38.033411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:38.036574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36a804-705f-4227-8efc-589f529ab786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:05:38.037274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36a804-705f-4227-8efc-589f529ab786 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:38.037790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36a804-705f-4227-8efc-589f529ab786 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:38.038135 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36a804-705f-4227-8efc-589f529ab786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:38.040420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3d36a804-705f-4227-8efc-589f529ab786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:38.159764 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36a804-705f-4227-8efc-589f529ab786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:05:38.161241 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36a804-705f-4227-8efc-589f529ab786 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:05:38.180804 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 648/1314] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:05:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:05:40.499017 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38732b6d-2316-42c8-ace6-5214d5d183a0 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:40.499567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38732b6d-2316-42c8-ace6-5214d5d183a0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:40.499567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38732b6d-2316-42c8-ace6-5214d5d183a0 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:40.501059 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 667/1315] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:05:40.506205 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34adfc9-8985-4172-8b22-e6ffe24cb4d0 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:40.506587 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34adfc9-8985-4172-8b22-e6ffe24cb4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:40.506782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34adfc9-8985-4172-8b22-e6ffe24cb4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:40.570099 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d34adfc9-8985-4172-8b22-e6ffe24cb4d0 None None] MFA Rules not processed for user `aabb9fdd0abd4b0e9c39fa9e7814c733`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:05:40.571324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d34adfc9-8985-4172-8b22-e6ffe24cb4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:40.621534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:40.665799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 649/1316] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:05:40 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:05:40.705267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38732b6d-2316-42c8-ace6-5214d5d183a0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:05:40.707312 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-38732b6d-2316-42c8-ace6-5214d5d183a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:40.716792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:40.810144 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38732b6d-2316-42c8-ace6-5214d5d183a0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:05:40.881894 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581f302-b1ee-4ab8-8697-ac2b87326704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:05:40.882535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581f302-b1ee-4ab8-8697-ac2b87326704 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:40.882814 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581f302-b1ee-4ab8-8697-ac2b87326704 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:40.883087 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581f302-b1ee-4ab8-8697-ac2b87326704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:40.884727 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9581f302-b1ee-4ab8-8697-ac2b87326704 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:40.945029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:40.990901 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9581f302-b1ee-4ab8-8697-ac2b87326704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:05:40.992334 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9581f302-b1ee-4ab8-8697-ac2b87326704 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:05:41.010159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 668/1317] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:05:40 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:05:43.550375 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1c255-c53e-4169-a479-0276a4b28394 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:43.550915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1c255-c53e-4169-a479-0276a4b28394 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:43.550915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c1c255-c53e-4169-a479-0276a4b28394 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:43.552257 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 650/1318] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:05:43.557362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09542369-eb19-4a28-bf7e-cb19bfb26a50 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:43.557745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09542369-eb19-4a28-bf7e-cb19bfb26a50 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:43.557909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09542369-eb19-4a28-bf7e-cb19bfb26a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:43.620829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-09542369-eb19-4a28-bf7e-cb19bfb26a50 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:05:43.622049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-09542369-eb19-4a28-bf7e-cb19bfb26a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:43.715106 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 669/1319] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:05:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:05:43.765212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd49bd8-70e4-45df-8a1f-ea9f302fb62f None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:43.765493 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd49bd8-70e4-45df-8a1f-ea9f302fb62f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:43.765767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd49bd8-70e4-45df-8a1f-ea9f302fb62f None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:43.767133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 651/1320] 10.222.0.61 () {54 vars in 964 bytes} [Thu Jul 14 10:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:05:43.773144 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d282757-3c9f-4cc8-94e0-880214e4ddbe None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:43.773430 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d282757-3c9f-4cc8-94e0-880214e4ddbe None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:43.773700 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d282757-3c9f-4cc8-94e0-880214e4ddbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:43.794313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:43.846710 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0d282757-3c9f-4cc8-94e0-880214e4ddbe None None] MFA Rules not processed for user `cd6d19c4406b4b228065b98a9a074b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:05:43.847829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0d282757-3c9f-4cc8-94e0-880214e4ddbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:43.919790 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 670/1321] 10.222.0.61 () {58 vars in 1061 bytes} [Thu Jul 14 10:05:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:05:43.929228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd49bd8-70e4-45df-8a1f-ea9f302fb62f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:05:43.930738 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1bd49bd8-70e4-45df-8a1f-ea9f302fb62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:44.033102 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd49bd8-70e4-45df-8a1f-ea9f302fb62f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:05:44.105035 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af91b493-fcc4-40bf-b82e-5276e6642462 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:05:44.105534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91b493-fcc4-40bf-b82e-5276e6642462 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:44.105782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91b493-fcc4-40bf-b82e-5276e6642462 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:44.106059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af91b493-fcc4-40bf-b82e-5276e6642462 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:44.107705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-af91b493-fcc4-40bf-b82e-5276e6642462 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:44.115687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:44.216489 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91b493-fcc4-40bf-b82e-5276e6642462 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:05:44.217954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af91b493-fcc4-40bf-b82e-5276e6642462 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:05:44.236534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 652/1322] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:05:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:05:47.350449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273a9ce-a480-4859-b30a-3e5aefa0256d None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:47.351013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273a9ce-a480-4859-b30a-3e5aefa0256d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:47.351013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b273a9ce-a480-4859-b30a-3e5aefa0256d None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:47.352444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 671/1323] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:05:47.357849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812cbb2-f101-4358-b560-f1e2364279c7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:47.358249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812cbb2-f101-4358-b560-f1e2364279c7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:47.358399 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a812cbb2-f101-4358-b560-f1e2364279c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:47.424554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a812cbb2-f101-4358-b560-f1e2364279c7 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:05:47.425792 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a812cbb2-f101-4358-b560-f1e2364279c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:47.481988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:47.526323 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 653/1324] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:05:47.563398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273a9ce-a480-4859-b30a-3e5aefa0256d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:05:47.565704 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b273a9ce-a480-4859-b30a-3e5aefa0256d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:47.676002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b273a9ce-a480-4859-b30a-3e5aefa0256d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:05:47.689484 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:47.754456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088970d-86dd-494f-b80f-57a57d87af2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:05:47.755105 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088970d-86dd-494f-b80f-57a57d87af2d service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:47.755387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088970d-86dd-494f-b80f-57a57d87af2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:47.755662 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088970d-86dd-494f-b80f-57a57d87af2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:47.757353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e088970d-86dd-494f-b80f-57a57d87af2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:47.866352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088970d-86dd-494f-b80f-57a57d87af2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:05:47.867717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088970d-86dd-494f-b80f-57a57d87af2d service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:05:47.885793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 672/1325] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:05:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:05:49.980174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09d996-363a-4a2f-8434-d5577fee8ab3 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:49.980681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09d996-363a-4a2f-8434-d5577fee8ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:49.980681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea09d996-363a-4a2f-8434-d5577fee8ab3 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:49.982048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 654/1326] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:05:49.987776 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff576a6b-ae44-4c44-9e47-94e31ef3cdec None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:49.988194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff576a6b-ae44-4c44-9e47-94e31ef3cdec None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:49.988467 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff576a6b-ae44-4c44-9e47-94e31ef3cdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:50.037531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:50.057013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ff576a6b-ae44-4c44-9e47-94e31ef3cdec None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:05:50.058227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ff576a6b-ae44-4c44-9e47-94e31ef3cdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:50.160471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 673/1327] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:05:50.196559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09d996-363a-4a2f-8434-d5577fee8ab3 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:05:50.198128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ea09d996-363a-4a2f-8434-d5577fee8ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:50.304576 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea09d996-363a-4a2f-8434-d5577fee8ab3 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:05:50.382120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966ca76-ba5e-4942-a80a-0611d385e7e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:05:50.382784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966ca76-ba5e-4942-a80a-0611d385e7e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:50.383065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966ca76-ba5e-4942-a80a-0611d385e7e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:50.383339 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966ca76-ba5e-4942-a80a-0611d385e7e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:50.385117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7966ca76-ba5e-4942-a80a-0611d385e7e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:50.405266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:50.497005 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966ca76-ba5e-4942-a80a-0611d385e7e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:05:50.498435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966ca76-ba5e-4942-a80a-0611d385e7e1 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:05:50.516721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 655/1328] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:05:52.611361 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a0953-54b7-4a13-9827-9ef86b11fbec None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:52.611909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a0953-54b7-4a13-9827-9ef86b11fbec None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:52.611909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a0953-54b7-4a13-9827-9ef86b11fbec None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:52.613460 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 674/1329] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:05:52.618544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57715028-cb26-47a0-84c9-137c5f5daff4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:52.618951 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57715028-cb26-47a0-84c9-137c5f5daff4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:52.619232 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57715028-cb26-47a0-84c9-137c5f5daff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:52.687649 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-57715028-cb26-47a0-84c9-137c5f5daff4 None None] MFA Rules not processed for user `aabb9fdd0abd4b0e9c39fa9e7814c733`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:05:52.688891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-57715028-cb26-47a0-84c9-137c5f5daff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:52.758479 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:52.791509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 656/1330] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:05:52 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:05:52.829401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a0953-54b7-4a13-9827-9ef86b11fbec None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:05:52.830969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-da0a0953-54b7-4a13-9827-9ef86b11fbec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:52.932104 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a0953-54b7-4a13-9827-9ef86b11fbec None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:05:52.953604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:53.006553 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d958a07-6384-4eeb-93b8-77f693535180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:05:53.007193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d958a07-6384-4eeb-93b8-77f693535180 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:53.007469 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d958a07-6384-4eeb-93b8-77f693535180 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:53.007737 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d958a07-6384-4eeb-93b8-77f693535180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:53.009401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7d958a07-6384-4eeb-93b8-77f693535180 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:53.122313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d958a07-6384-4eeb-93b8-77f693535180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:05:53.123858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d958a07-6384-4eeb-93b8-77f693535180 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:05:53.150910 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 675/1331] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:05:52 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:05:55.448894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d66b32-dd06-4c61-a232-1013279d9bf6 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:55.449468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d66b32-dd06-4c61-a232-1013279d9bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:55.449468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d66b32-dd06-4c61-a232-1013279d9bf6 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:55.451611 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 657/1332] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:05:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:05:55.457457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a6e30-87c3-4bc1-8950-c39fa18d6f2c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:55.457870 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a6e30-87c3-4bc1-8950-c39fa18d6f2c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:55.458005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0a6e30-87c3-4bc1-8950-c39fa18d6f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:55.518338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:55.521368 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4e0a6e30-87c3-4bc1-8950-c39fa18d6f2c None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:05:55.523111 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4e0a6e30-87c3-4bc1-8950-c39fa18d6f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:55.615005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 676/1333] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:05:55.650827 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d66b32-dd06-4c61-a232-1013279d9bf6 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:05:55.652331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e2d66b32-dd06-4c61-a232-1013279d9bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:55.753849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d66b32-dd06-4c61-a232-1013279d9bf6 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:05:55.825705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5776db2e-6655-4a27-aa88-7f8fda4113cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:05:55.826283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776db2e-6655-4a27-aa88-7f8fda4113cd service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:55.826564 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776db2e-6655-4a27-aa88-7f8fda4113cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:55.826839 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5776db2e-6655-4a27-aa88-7f8fda4113cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:55.828472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5776db2e-6655-4a27-aa88-7f8fda4113cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:55.855218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:55.936004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5776db2e-6655-4a27-aa88-7f8fda4113cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:05:55.937464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5776db2e-6655-4a27-aa88-7f8fda4113cd service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:05:55.955812 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 658/1334] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:05:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:05:58.032782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a14639-f23a-481d-9194-f2cac4e75827 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:58.033387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a14639-f23a-481d-9194-f2cac4e75827 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:58.033387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a14639-f23a-481d-9194-f2cac4e75827 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:58.034720 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 677/1335] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:05:58.040013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3ba17-4213-478f-941a-1409a01474b0 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:58.040704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3ba17-4213-478f-941a-1409a01474b0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:58.040827 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3ba17-4213-478f-941a-1409a01474b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:58.104362 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2be3ba17-4213-478f-941a-1409a01474b0 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:05:58.105587 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2be3ba17-4213-478f-941a-1409a01474b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:58.181725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:58.205941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 659/1336] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:05:58.245551 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a14639-f23a-481d-9194-f2cac4e75827 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:05:58.247166 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d6a14639-f23a-481d-9194-f2cac4e75827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:58.349734 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a14639-f23a-481d-9194-f2cac4e75827 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:05:58.377883 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:05:58.423171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e589c-aaae-4845-9a6b-643eb020534e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:05:58.423845 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e589c-aaae-4845-9a6b-643eb020534e service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:05:58.424203 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e589c-aaae-4845-9a6b-643eb020534e service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:05:58.424431 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e589c-aaae-4845-9a6b-643eb020534e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:05:58.426148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fe0e589c-aaae-4845-9a6b-643eb020534e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:05:58.531235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e589c-aaae-4845-9a6b-643eb020534e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:05:58.532652 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e589c-aaae-4845-9a6b-643eb020534e service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:05:58.551617 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 678/1337] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:06:00.985622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1291b0-7163-4488-a0dc-c09d12d75557 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:00.986753 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1291b0-7163-4488-a0dc-c09d12d75557 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:00.986753 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1291b0-7163-4488-a0dc-c09d12d75557 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:00.988064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 660/1338] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:00.993364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d890902-89f7-4332-aeb5-ee35f6e7ff10 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:00.993791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d890902-89f7-4332-aeb5-ee35f6e7ff10 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:00.993907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d890902-89f7-4332-aeb5-ee35f6e7ff10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:01.057402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8d890902-89f7-4332-aeb5-ee35f6e7ff10 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:01.058619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8d890902-89f7-4332-aeb5-ee35f6e7ff10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:01.067467 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:01.153950 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 679/1339] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:00 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:01.189599 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1291b0-7163-4488-a0dc-c09d12d75557 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:01.191122 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2e1291b0-7163-4488-a0dc-c09d12d75557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:01.292158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1291b0-7163-4488-a0dc-c09d12d75557 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:01.365948 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2f75d-2bc3-4443-8464-54f6efc4a7cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:01.366615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2f75d-2bc3-4443-8464-54f6efc4a7cd service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:01.366898 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2f75d-2bc3-4443-8464-54f6efc4a7cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:01.367173 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2f75d-2bc3-4443-8464-54f6efc4a7cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:01.368854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-46b2f75d-2bc3-4443-8464-54f6efc4a7cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:01.405586 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:01.478682 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2f75d-2bc3-4443-8464-54f6efc4a7cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:01.480111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b2f75d-2bc3-4443-8464-54f6efc4a7cd service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:01.498593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 661/1340] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:05.560177 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa3cee-e542-4a38-81c5-8205d3b042a5 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:05.560748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa3cee-e542-4a38-81c5-8205d3b042a5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:05.560748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62aa3cee-e542-4a38-81c5-8205d3b042a5 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:05.562120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 680/1341] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:05.568107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa3a5-41fc-4c73-adbf-1fc27f8359c0 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:05.568538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa3a5-41fc-4c73-adbf-1fc27f8359c0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:05.568662 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398fa3a5-41fc-4c73-adbf-1fc27f8359c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:05.633742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-398fa3a5-41fc-4c73-adbf-1fc27f8359c0 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:05.634852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-398fa3a5-41fc-4c73-adbf-1fc27f8359c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:05.719174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:05.731219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 662/1342] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:05.769275 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa3cee-e542-4a38-81c5-8205d3b042a5 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:05.770797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-62aa3cee-e542-4a38-81c5-8205d3b042a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:05.872471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62aa3cee-e542-4a38-81c5-8205d3b042a5 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:05.912273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:05.949538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa3ed39-a81d-4f3c-aaa5-dee07058c5fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:05.950189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3ed39-a81d-4f3c-aaa5-dee07058c5fd service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:05.950473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3ed39-a81d-4f3c-aaa5-dee07058c5fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:05.950742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa3ed39-a81d-4f3c-aaa5-dee07058c5fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:05.952387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0aa3ed39-a81d-4f3c-aaa5-dee07058c5fd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:06.061245 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3ed39-a81d-4f3c-aaa5-dee07058c5fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:06.062680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa3ed39-a81d-4f3c-aaa5-dee07058c5fd service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:06.081895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 681/1343] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:08.744459 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3741849-1faa-4d10-a71d-b9e8f3a719a2 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:08.745076 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3741849-1faa-4d10-a71d-b9e8f3a719a2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:08.745076 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3741849-1faa-4d10-a71d-b9e8f3a719a2 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:08.746595 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 663/1344] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:08.752863 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82e62f-b5a1-4fec-8796-6b156572da41 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:08.753395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82e62f-b5a1-4fec-8796-6b156572da41 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:08.753513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82e62f-b5a1-4fec-8796-6b156572da41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:08.828963 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6e82e62f-b5a1-4fec-8796-6b156572da41 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:08.830912 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6e82e62f-b5a1-4fec-8796-6b156572da41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:08.850481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:08.938545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 682/1345] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:08.976504 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3741849-1faa-4d10-a71d-b9e8f3a719a2 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:08.978049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d3741849-1faa-4d10-a71d-b9e8f3a719a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:09.103130 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3741849-1faa-4d10-a71d-b9e8f3a719a2 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:09.178815 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd77690-be8b-4273-9aee-d2382cd8f191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:09.179459 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd77690-be8b-4273-9aee-d2382cd8f191 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:09.179737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd77690-be8b-4273-9aee-d2382cd8f191 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:09.180004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd77690-be8b-4273-9aee-d2382cd8f191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:09.181680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cbd77690-be8b-4273-9aee-d2382cd8f191 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:09.224222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:09.293292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd77690-be8b-4273-9aee-d2382cd8f191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:09.294729 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd77690-be8b-4273-9aee-d2382cd8f191 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:09.312827 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 664/1346] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:08 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:13.271194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced6191-e194-4245-b07f-c8f3117ea4ae None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:13.271695 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced6191-e194-4245-b07f-c8f3117ea4ae None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:13.271803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced6191-e194-4245-b07f-c8f3117ea4ae None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:13.273160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 683/1347] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:13.278249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b615d3e-915e-4530-af87-60ce5bb02a37 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:13.278660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b615d3e-915e-4530-af87-60ce5bb02a37 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:13.278801 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b615d3e-915e-4530-af87-60ce5bb02a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:13.345654 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-1b615d3e-915e-4530-af87-60ce5bb02a37 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:13.346864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1b615d3e-915e-4530-af87-60ce5bb02a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:13.441634 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:13.444344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 665/1348] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:13.482116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced6191-e194-4245-b07f-c8f3117ea4ae None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:13.484068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bced6191-e194-4245-b07f-c8f3117ea4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:13.609600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced6191-e194-4245-b07f-c8f3117ea4ae None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:13.661094 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:13.689874 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96928e-0882-41a7-abad-11b7f97e23a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:13.690536 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96928e-0882-41a7-abad-11b7f97e23a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:13.690818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96928e-0882-41a7-abad-11b7f97e23a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:13.691093 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96928e-0882-41a7-abad-11b7f97e23a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:13.692785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8e96928e-0882-41a7-abad-11b7f97e23a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:13.807063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96928e-0882-41a7-abad-11b7f97e23a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:13.808410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96928e-0882-41a7-abad-11b7f97e23a3 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:13.827558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 684/1349] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:18.739011 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0468a7-31f6-47e0-9744-2d4a03bad9c9 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:18.740347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0468a7-31f6-47e0-9744-2d4a03bad9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:18.741242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0468a7-31f6-47e0-9744-2d4a03bad9c9 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:18.745347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 666/1350] 10.222.0.61 () {54 vars in 921 bytes} [Thu Jul 14 10:06:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:18.749502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acea04-be59-4bcf-bebb-ae954eba3678 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:18.750217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acea04-be59-4bcf-bebb-ae954eba3678 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:18.751365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02acea04-be59-4bcf-bebb-ae954eba3678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:18.885244 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-02acea04-be59-4bcf-bebb-ae954eba3678 None None] MFA Rules not processed for user `ff4c42a4f4f549378e5bc1cd92f801c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:18.886573 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-02acea04-be59-4bcf-bebb-ae954eba3678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:18.929330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:19.004867 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 685/1351] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Jul 14 10:06:18 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:23.092205 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005169b-677d-4c70-a87c-a1fe85572fc7 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:23.093166 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005169b-677d-4c70-a87c-a1fe85572fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:23.093482 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0005169b-677d-4c70-a87c-a1fe85572fc7 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:23.094912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 667/1352] 10.222.0.61 () {54 vars in 921 bytes} [Thu Jul 14 10:06:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:23.100166 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070a57e-919d-43a3-b7e5-8e639fd962aa None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:23.100962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070a57e-919d-43a3-b7e5-8e639fd962aa None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:23.101447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070a57e-919d-43a3-b7e5-8e639fd962aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:23.179118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3070a57e-919d-43a3-b7e5-8e639fd962aa None None] MFA Rules not processed for user `ff4c42a4f4f549378e5bc1cd92f801c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:23.180306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3070a57e-919d-43a3-b7e5-8e639fd962aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:23.268362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:23.306389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 686/1353] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Jul 14 10:06:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:24.722512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9317f-f912-41ff-8bdd-b18cd63953a2 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:24.723088 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9317f-f912-41ff-8bdd-b18cd63953a2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:24.723339 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9317f-f912-41ff-8bdd-b18cd63953a2 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:24.724981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 668/1354] 10.222.0.61 () {54 vars in 921 bytes} [Thu Jul 14 10:06:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:24.730100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b81a7-09f8-40a8-b43e-f7a4934faf77 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:24.730518 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b81a7-09f8-40a8-b43e-f7a4934faf77 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:24.730790 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474b81a7-09f8-40a8-b43e-f7a4934faf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:24.804572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-474b81a7-09f8-40a8-b43e-f7a4934faf77 None None] MFA Rules not processed for user `ff4c42a4f4f549378e5bc1cd92f801c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:24.806153 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-474b81a7-09f8-40a8-b43e-f7a4934faf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:24.917649 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 687/1355] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Jul 14 10:06:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:06:28.314391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c35637-c358-4785-85e7-e16afe00fcf2 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:28.314882 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c35637-c358-4785-85e7-e16afe00fcf2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:28.314882 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c35637-c358-4785-85e7-e16afe00fcf2 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:28.316215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 669/1356] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:28.321856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3b8d5-b8d9-41c6-9a19-5acbf532dca5 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:28.322344 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3b8d5-b8d9-41c6-9a19-5acbf532dca5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:28.324226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a3b8d5-b8d9-41c6-9a19-5acbf532dca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:28.368255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:28.406770 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-89a3b8d5-b8d9-41c6-9a19-5acbf532dca5 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:28.408727 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-89a3b8d5-b8d9-41c6-9a19-5acbf532dca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:28.516689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 688/1357] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:28.570262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e451b-adbb-40f5-bd7d-1e208a16d216 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:28.571295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e451b-adbb-40f5-bd7d-1e208a16d216 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:28.572003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19e451b-adbb-40f5-bd7d-1e208a16d216 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:28.574154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 670/1358] 10.222.0.61 () {54 vars in 961 bytes} [Thu Jul 14 10:06:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:28.582466 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a21f62-5652-4bdb-aa94-da58c3c23daf None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:28.583440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a21f62-5652-4bdb-aa94-da58c3c23daf None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:28.584062 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a21f62-5652-4bdb-aa94-da58c3c23daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:28.671617 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-05a21f62-5652-4bdb-aa94-da58c3c23daf None None] MFA Rules not processed for user `e9f4efa3e2f343c2ac858a277601a7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:28.673006 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-05a21f62-5652-4bdb-aa94-da58c3c23daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:28.723955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:28.798206 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 689/1359] 10.222.0.61 () {58 vars in 1058 bytes} [Thu Jul 14 10:06:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:06:28.806008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e451b-adbb-40f5-bd7d-1e208a16d216 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:28.807556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d19e451b-adbb-40f5-bd7d-1e208a16d216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:28.962328 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19e451b-adbb-40f5-bd7d-1e208a16d216 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:29.047881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:29.091106 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ae68f-9c83-4933-80b4-a3d659e861e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:29.091754 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ae68f-9c83-4933-80b4-a3d659e861e0 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:29.092033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ae68f-9c83-4933-80b4-a3d659e861e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:29.092303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ae68f-9c83-4933-80b4-a3d659e861e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:29.094110 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cc6ae68f-9c83-4933-80b4-a3d659e861e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:29.188651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ae68f-9c83-4933-80b4-a3d659e861e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:29.190422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ae68f-9c83-4933-80b4-a3d659e861e0 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:29.210224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 671/1360] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:31.461597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a2cfa-1728-4a45-8d8c-00ab1c977889 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:31.462167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a2cfa-1728-4a45-8d8c-00ab1c977889 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:31.462167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a2cfa-1728-4a45-8d8c-00ab1c977889 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:31.463677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 690/1361] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:31.469359 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379119dd-2104-4a91-83ef-425838402fbe None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:31.469817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379119dd-2104-4a91-83ef-425838402fbe None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:31.469905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379119dd-2104-4a91-83ef-425838402fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:31.542531 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-379119dd-2104-4a91-83ef-425838402fbe None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:31.543741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-379119dd-2104-4a91-83ef-425838402fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:31.583518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:31.651641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 672/1362] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:31.705574 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a2cfa-1728-4a45-8d8c-00ab1c977889 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:31.707091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d38a2cfa-1728-4a45-8d8c-00ab1c977889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:31.879333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38a2cfa-1728-4a45-8d8c-00ab1c977889 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:31.890017 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:32.045000 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abd12a-133b-480a-9945-7572235ba7b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:32.046085 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abd12a-133b-480a-9945-7572235ba7b9 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:32.046600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abd12a-133b-480a-9945-7572235ba7b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:32.047454 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abd12a-133b-480a-9945-7572235ba7b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:32.048766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-09abd12a-133b-480a-9945-7572235ba7b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:32.144530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abd12a-133b-480a-9945-7572235ba7b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:32.146264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abd12a-133b-480a-9945-7572235ba7b9 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:32.165643 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 691/1363] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:34.568356 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490a222-2418-49d7-be0c-296d1ff0c368 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:34.569036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490a222-2418-49d7-be0c-296d1ff0c368 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:34.569036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490a222-2418-49d7-be0c-296d1ff0c368 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:34.570486 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 673/1364] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:34.576076 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef21cd1-359f-4988-bcf9-da8d7a8eba15 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:34.576555 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef21cd1-359f-4988-bcf9-da8d7a8eba15 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:34.576655 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef21cd1-359f-4988-bcf9-da8d7a8eba15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:34.591889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:34.650365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-aef21cd1-359f-4988-bcf9-da8d7a8eba15 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:34.651568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-aef21cd1-359f-4988-bcf9-da8d7a8eba15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:34.757152 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 692/1365] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:34.801387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bd75c-7737-4612-ad38-126c26d3029a None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:34.801864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bd75c-7737-4612-ad38-126c26d3029a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:34.802217 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080bd75c-7737-4612-ad38-126c26d3029a None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:34.803701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 674/1366] 10.222.0.61 () {54 vars in 961 bytes} [Thu Jul 14 10:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:34.809000 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ba282-e30a-4ecb-b40e-2f2de70f605f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:34.809294 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ba282-e30a-4ecb-b40e-2f2de70f605f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:34.809572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ba282-e30a-4ecb-b40e-2f2de70f605f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:34.887593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-401ba282-e30a-4ecb-b40e-2f2de70f605f None None] MFA Rules not processed for user `e9f4efa3e2f343c2ac858a277601a7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:34.888708 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-401ba282-e30a-4ecb-b40e-2f2de70f605f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:34.932167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:35.023355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 693/1367] 10.222.0.61 () {58 vars in 1058 bytes} [Thu Jul 14 10:06:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:35.032136 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080bd75c-7737-4612-ad38-126c26d3029a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:35.033897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-080bd75c-7737-4612-ad38-126c26d3029a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:35.192500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080bd75c-7737-4612-ad38-126c26d3029a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:35.213970 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:35.320178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4abee0-09e8-4e96-8435-33ebcb543e0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:35.320832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4abee0-09e8-4e96-8435-33ebcb543e0c service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:35.321157 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4abee0-09e8-4e96-8435-33ebcb543e0c service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:35.321434 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4abee0-09e8-4e96-8435-33ebcb543e0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:35.323096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-af4abee0-09e8-4e96-8435-33ebcb543e0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:35.418148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4abee0-09e8-4e96-8435-33ebcb543e0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:35.419868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4abee0-09e8-4e96-8435-33ebcb543e0c service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:35.439609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 675/1368] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:06:36.461952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5ad71-7330-48b8-b0df-a160cdacb201 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:36.462461 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5ad71-7330-48b8-b0df-a160cdacb201 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:36.462573 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5ad71-7330-48b8-b0df-a160cdacb201 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:36.464072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 694/1369] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:36.469419 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c5a29-ca5d-4069-af89-eb9301c1766b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:36.470249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c5a29-ca5d-4069-af89-eb9301c1766b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:36.470749 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527c5a29-ca5d-4069-af89-eb9301c1766b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:36.516736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:36.559977 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-527c5a29-ca5d-4069-af89-eb9301c1766b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:36.561210 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-527c5a29-ca5d-4069-af89-eb9301c1766b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:36.682083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 676/1370] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:37.024511 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e748bf-bee8-4800-bdb8-ec90d06c0812 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:37.024511 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e748bf-bee8-4800-bdb8-ec90d06c0812 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:37.024511 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e748bf-bee8-4800-bdb8-ec90d06c0812 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:37.025932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 695/1371] 10.222.0.61 () {54 vars in 963 bytes} [Thu Jul 14 10:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:37.029966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f99e48-2abd-4714-9131-f0b9aedeb9b4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:37.030215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f99e48-2abd-4714-9131-f0b9aedeb9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:37.030632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f99e48-2abd-4714-9131-f0b9aedeb9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:37.114028 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f9f99e48-2abd-4714-9131-f0b9aedeb9b4 None None] MFA Rules not processed for user `52e3fbd0400a49a69932ce426218f4a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:37.115243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f9f99e48-2abd-4714-9131-f0b9aedeb9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:37.163594 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:37.197350 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 677/1372] 10.222.0.61 () {58 vars in 1060 bytes} [Thu Jul 14 10:06:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:37.205400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e748bf-bee8-4800-bdb8-ec90d06c0812 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:37.206915 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-21e748bf-bee8-4800-bdb8-ec90d06c0812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:37.332087 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e748bf-bee8-4800-bdb8-ec90d06c0812 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:37.373654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:37.411711 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0f333-17ab-4278-9165-2f02f10f3b4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:37.412376 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0f333-17ab-4278-9165-2f02f10f3b4d service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:37.412661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0f333-17ab-4278-9165-2f02f10f3b4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:37.412947 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d0f333-17ab-4278-9165-2f02f10f3b4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:37.414921 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-36d0f333-17ab-4278-9165-2f02f10f3b4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:37.533278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0f333-17ab-4278-9165-2f02f10f3b4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:37.535011 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d0f333-17ab-4278-9165-2f02f10f3b4d service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:37.561684 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d2700-82cf-419a-9904-e1fc0288a385 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:37.562316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d2700-82cf-419a-9904-e1fc0288a385 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:37.562316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d2700-82cf-419a-9904-e1fc0288a385 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:37.563907 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 678/1373] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:06:37.565289 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 696/1374] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:37.568236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25b9d6-7ad4-4190-8196-c577e0dffe75 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:37.568562 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25b9d6-7ad4-4190-8196-c577e0dffe75 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:37.568856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d25b9d6-7ad4-4190-8196-c577e0dffe75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:37.633888 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0d25b9d6-7ad4-4190-8196-c577e0dffe75 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:37.635195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0d25b9d6-7ad4-4190-8196-c577e0dffe75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:37.737215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 679/1375] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:37.781506 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0f333-17ab-4278-9165-2f02f10f3b4d service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:37.782708 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-36d0f333-17ab-4278-9165-2f02f10f3b4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:37.864903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:37.907811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d0f333-17ab-4278-9165-2f02f10f3b4d service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:38.020479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bab93-9e84-4b64-acca-0ff1415dfda5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:38.021539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bab93-9e84-4b64-acca-0ff1415dfda5 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:38.021694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bab93-9e84-4b64-acca-0ff1415dfda5 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:38.022051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bab93-9e84-4b64-acca-0ff1415dfda5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:38.024167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b02bab93-9e84-4b64-acca-0ff1415dfda5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:38.103267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:38.114605 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bab93-9e84-4b64-acca-0ff1415dfda5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:38.116193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bab93-9e84-4b64-acca-0ff1415dfda5 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:38.134339 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 697/1376] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:40.180172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d677aa6-1434-454e-a7ba-a0f0fe313122 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:40.180677 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d677aa6-1434-454e-a7ba-a0f0fe313122 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:40.180677 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d677aa6-1434-454e-a7ba-a0f0fe313122 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:40.182088 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 680/1377] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:40.187937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d68a18-14cd-49fe-b072-0aa1acad37e8 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:40.188346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d68a18-14cd-49fe-b072-0aa1acad37e8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:40.188489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d68a18-14cd-49fe-b072-0aa1acad37e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:40.257315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c5d68a18-14cd-49fe-b072-0aa1acad37e8 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:40.258517 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c5d68a18-14cd-49fe-b072-0aa1acad37e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:40.366885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 698/1378] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:40.411001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d677aa6-1434-454e-a7ba-a0f0fe313122 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:40.412514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9d677aa6-1434-454e-a7ba-a0f0fe313122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:40.469401 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:40.591691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d677aa6-1434-454e-a7ba-a0f0fe313122 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:40.703713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd261f29-92ce-445b-9345-fbf47b36b4b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:40.704351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd261f29-92ce-445b-9345-fbf47b36b4b9 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:40.704635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd261f29-92ce-445b-9345-fbf47b36b4b9 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:40.704900 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd261f29-92ce-445b-9345-fbf47b36b4b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:40.706536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cd261f29-92ce-445b-9345-fbf47b36b4b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:40.717762 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:40.802043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd261f29-92ce-445b-9345-fbf47b36b4b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:40.803635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd261f29-92ce-445b-9345-fbf47b36b4b9 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:40.822198 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 681/1379] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:42.882130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf8fda-f382-4207-ab81-8931c53834bf None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:42.882650 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf8fda-f382-4207-ab81-8931c53834bf None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:42.882650 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbf8fda-f382-4207-ab81-8931c53834bf None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:42.884195 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 699/1380] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:42.889373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6faefd-0af1-4b02-a78f-b3a054732389 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:42.890405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6faefd-0af1-4b02-a78f-b3a054732389 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:42.890596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6faefd-0af1-4b02-a78f-b3a054732389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:42.961816 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-1c6faefd-0af1-4b02-a78f-b3a054732389 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:42.963154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1c6faefd-0af1-4b02-a78f-b3a054732389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:43.066613 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:43.080033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 682/1381] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:43.127867 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf8fda-f382-4207-ab81-8931c53834bf None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:43.129466 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8fbf8fda-f382-4207-ab81-8931c53834bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:43.194377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:43.284307 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbf8fda-f382-4207-ab81-8931c53834bf None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:43.404058 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60157bee-94bb-4630-a43c-f2bb25b5fb28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:43.405214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60157bee-94bb-4630-a43c-f2bb25b5fb28 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:43.405545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60157bee-94bb-4630-a43c-f2bb25b5fb28 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:43.406053 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60157bee-94bb-4630-a43c-f2bb25b5fb28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:43.408172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-60157bee-94bb-4630-a43c-f2bb25b5fb28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:43.434203 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:43.506492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60157bee-94bb-4630-a43c-f2bb25b5fb28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:43.508078 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60157bee-94bb-4630-a43c-f2bb25b5fb28 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:43.527353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 700/1382] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:45.605633 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0accec-8a81-42e3-83e6-019d2c604a3c None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:45.606154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0accec-8a81-42e3-83e6-019d2c604a3c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:45.606154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0accec-8a81-42e3-83e6-019d2c604a3c None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:45.607586 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 683/1383] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:45.613265 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14eff7-43eb-47bf-8567-1eea7a0955c6 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:45.613762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14eff7-43eb-47bf-8567-1eea7a0955c6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:45.613916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f14eff7-43eb-47bf-8567-1eea7a0955c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:45.681313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7f14eff7-43eb-47bf-8567-1eea7a0955c6 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:45.682581 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7f14eff7-43eb-47bf-8567-1eea7a0955c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:45.778920 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:45.783498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 701/1384] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:45.826123 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0accec-8a81-42e3-83e6-019d2c604a3c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:45.827702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0b0accec-8a81-42e3-83e6-019d2c604a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:45.985786 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0accec-8a81-42e3-83e6-019d2c604a3c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:46.053631 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:46.107258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b27a0f-e326-4358-a9e8-1f45691ee0e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:46.107846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b27a0f-e326-4358-a9e8-1f45691ee0e7 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:46.108125 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b27a0f-e326-4358-a9e8-1f45691ee0e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:46.108396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b27a0f-e326-4358-a9e8-1f45691ee0e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:46.110074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a4b27a0f-e326-4358-a9e8-1f45691ee0e7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:46.203109 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b27a0f-e326-4358-a9e8-1f45691ee0e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:46.204709 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b27a0f-e326-4358-a9e8-1f45691ee0e7 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:46.223267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 684/1385] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:48.264958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f46cbf-604b-4fa4-a131-0e18128d896f None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:48.265500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f46cbf-604b-4fa4-a131-0e18128d896f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:48.265500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f46cbf-604b-4fa4-a131-0e18128d896f None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:48.266934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 702/1386] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:48.272392 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76d128-1daa-410a-8ee5-b242ad95f53a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:48.272894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76d128-1daa-410a-8ee5-b242ad95f53a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:48.273078 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d76d128-1daa-410a-8ee5-b242ad95f53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:48.340597 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6d76d128-1daa-410a-8ee5-b242ad95f53a None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:48.341998 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6d76d128-1daa-410a-8ee5-b242ad95f53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:48.368813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:48.443539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 685/1387] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:48.489147 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f46cbf-604b-4fa4-a131-0e18128d896f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:48.491031 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-88f46cbf-604b-4fa4-a131-0e18128d896f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:48.669330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f46cbf-604b-4fa4-a131-0e18128d896f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:48.739310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:48.804411 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21466519-b30f-4303-9165-23c09b57b8a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:48.805593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21466519-b30f-4303-9165-23c09b57b8a2 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:48.806087 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21466519-b30f-4303-9165-23c09b57b8a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:48.806590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21466519-b30f-4303-9165-23c09b57b8a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:48.809964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-21466519-b30f-4303-9165-23c09b57b8a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:48.929158 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21466519-b30f-4303-9165-23c09b57b8a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:48.930823 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21466519-b30f-4303-9165-23c09b57b8a2 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:48.952468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 703/1388] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:51.057708 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb90461-17c7-4a56-82ad-0c76596fa18c None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:51.058272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb90461-17c7-4a56-82ad-0c76596fa18c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:51.058272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb90461-17c7-4a56-82ad-0c76596fa18c None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:51.059644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 686/1389] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:51.064896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a127ae3-2bc7-48c9-9386-b181d5412790 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:51.065375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a127ae3-2bc7-48c9-9386-b181d5412790 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:51.065477 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a127ae3-2bc7-48c9-9386-b181d5412790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:51.129548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1a127ae3-2bc7-48c9-9386-b181d5412790 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:51.130761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1a127ae3-2bc7-48c9-9386-b181d5412790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:51.165404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:51.227896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 704/1390] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:51.272052 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb90461-17c7-4a56-82ad-0c76596fa18c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:51.273681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-efb90461-17c7-4a56-82ad-0c76596fa18c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:51.420515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb90461-17c7-4a56-82ad-0c76596fa18c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:51.429718 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:51.533443 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a62be24-d05c-4131-a7d1-f829ef5d5b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:51.534083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62be24-d05c-4131-a7d1-f829ef5d5b11 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:51.534360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62be24-d05c-4131-a7d1-f829ef5d5b11 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:51.534626 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a62be24-d05c-4131-a7d1-f829ef5d5b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:51.536253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5a62be24-d05c-4131-a7d1-f829ef5d5b11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:51.625699 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62be24-d05c-4131-a7d1-f829ef5d5b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:51.627239 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a62be24-d05c-4131-a7d1-f829ef5d5b11 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:51.647388 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 687/1391] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:52.648407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df41ea-eb51-4843-a427-094bf734a5a3 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:52.648957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df41ea-eb51-4843-a427-094bf734a5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:52.649054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df41ea-eb51-4843-a427-094bf734a5a3 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:52.650456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 705/1392] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:52.656461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa827f-ad6b-4eb6-89e8-60ca7f6ee0ff None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:52.656910 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa827f-ad6b-4eb6-89e8-60ca7f6ee0ff None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:52.657103 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa827f-ad6b-4eb6-89e8-60ca7f6ee0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:52.671621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:52.723236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-72aa827f-ad6b-4eb6-89e8-60ca7f6ee0ff None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:52.724432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-72aa827f-ad6b-4eb6-89e8-60ca7f6ee0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:52.822537 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 688/1393] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:52.882276 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571b5ee-c343-4c67-b986-cd52da4c3a5a None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:52.882565 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571b5ee-c343-4c67-b986-cd52da4c3a5a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:52.882848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a571b5ee-c343-4c67-b986-cd52da4c3a5a None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:52.884240 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 706/1394] 10.222.0.61 () {54 vars in 962 bytes} [Thu Jul 14 10:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:52.889330 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b77ab-0149-48ce-8e08-e55ef4fdfcb6 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:52.889621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b77ab-0149-48ce-8e08-e55ef4fdfcb6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:52.890408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b77ab-0149-48ce-8e08-e55ef4fdfcb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:52.964909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-130b77ab-0149-48ce-8e08-e55ef4fdfcb6 None None] MFA Rules not processed for user `e3b5660cfd344dcd8abe14c88df686e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:52.966120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-130b77ab-0149-48ce-8e08-e55ef4fdfcb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:53.000664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:53.042587 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 689/1395] 10.222.0.61 () {58 vars in 1059 bytes} [Thu Jul 14 10:06:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:53.050189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571b5ee-c343-4c67-b986-cd52da4c3a5a None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:53.051853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a571b5ee-c343-4c67-b986-cd52da4c3a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:53.155096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a571b5ee-c343-4c67-b986-cd52da4c3a5a None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:53.216690 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:53.229249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6e0c0e-05a7-4b11-aef3-dbc681c7a7e3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b5660cfd344dcd8abe14c88df686e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:53.229906 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e0c0e-05a7-4b11-aef3-dbc681c7a7e3 service glance] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:53.230189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e0c0e-05a7-4b11-aef3-dbc681c7a7e3 service glance] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:53.230465 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6e0c0e-05a7-4b11-aef3-dbc681c7a7e3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:53.232121 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9b6e0c0e-05a7-4b11-aef3-dbc681c7a7e3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:53.347701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e0c0e-05a7-4b11-aef3-dbc681c7a7e3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:53.349207 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6e0c0e-05a7-4b11-aef3-dbc681c7a7e3 service glance] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:53.368045 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 707/1396] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:53.703821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36e700-727c-4e0c-9662-7fca722b1c7a None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:53.704394 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36e700-727c-4e0c-9662-7fca722b1c7a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:53.704394 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd36e700-727c-4e0c-9662-7fca722b1c7a None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:53.714595 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 690/1397] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:53 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:53.714698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e3690-f607-42cc-aa13-b6044d45dae2 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:53.714698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e3690-f607-42cc-aa13-b6044d45dae2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:53.714698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e3690-f607-42cc-aa13-b6044d45dae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:53.782626 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-666e3690-f607-42cc-aa13-b6044d45dae2 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:53.783750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-666e3690-f607-42cc-aa13-b6044d45dae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:53.826002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:53.883001 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 708/1398] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:53.929258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36e700-727c-4e0c-9662-7fca722b1c7a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:53.930833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-bd36e700-727c-4e0c-9662-7fca722b1c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:54.075753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891226b5-9c46-4a28-9ab6-ed940736e3d4 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:54.076080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891226b5-9c46-4a28-9ab6-ed940736e3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:54.076330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891226b5-9c46-4a28-9ab6-ed940736e3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:54.090307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd36e700-727c-4e0c-9662-7fca722b1c7a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:54.131486 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:54.141545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-891226b5-9c46-4a28-9ab6-ed940736e3d4 None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:54.142981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-891226b5-9c46-4a28-9ab6-ed940736e3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:54.208215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f9790-f808-455e-964b-6323f2ea81db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:54.208877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f9790-f808-455e-964b-6323f2ea81db service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:54.209185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f9790-f808-455e-964b-6323f2ea81db service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:54.209464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635f9790-f808-455e-964b-6323f2ea81db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:54.211092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-635f9790-f808-455e-964b-6323f2ea81db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:54.230431 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 709/1399] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Jul 14 10:06:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:06:54.279210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b282d3b-d235-4729-a624-70d244b08e3d None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:54.279490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b282d3b-d235-4729-a624-70d244b08e3d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:54.279756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b282d3b-d235-4729-a624-70d244b08e3d None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:54.283136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 710/1400] 10.222.0.61 () {54 vars in 942 bytes} [Thu Jul 14 10:06:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:06:54.289531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1935e3-15d8-4ab7-9068-0f84a82fc345 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:54.289815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1935e3-15d8-4ab7-9068-0f84a82fc345 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:54.290088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1935e3-15d8-4ab7-9068-0f84a82fc345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:54.305498 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f9790-f808-455e-964b-6323f2ea81db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:54.307088 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635f9790-f808-455e-964b-6323f2ea81db service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:54.326467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 691/1401] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:54.333708 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:54.375792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-bb1935e3-15d8-4ab7-9068-0f84a82fc345 None None] MFA Rules not processed for user `d5fa502e8b014c719fcece0a130fdd7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:54.377099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bb1935e3-15d8-4ab7-9068-0f84a82fc345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:54.493393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 711/1402] 10.222.0.61 () {58 vars in 1039 bytes} [Thu Jul 14 10:06:54 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:54.503789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f9790-f808-455e-964b-6323f2ea81db service nova] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:54.505148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-635f9790-f808-455e-964b-6323f2ea81db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:54.556318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:54.599694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635f9790-f808-455e-964b-6323f2ea81db service nova] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:54.703245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e014ea7f-f062-4bd4-94b7-8aaa42c9b122 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:54.703954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014ea7f-f062-4bd4-94b7-8aaa42c9b122 service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:54.704484 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014ea7f-f062-4bd4-94b7-8aaa42c9b122 service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:54.704484 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e014ea7f-f062-4bd4-94b7-8aaa42c9b122 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:54.706211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e014ea7f-f062-4bd4-94b7-8aaa42c9b122 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:54.787732 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:54.792406 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014ea7f-f062-4bd4-94b7-8aaa42c9b122 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:54.794029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e014ea7f-f062-4bd4-94b7-8aaa42c9b122 service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:54.802895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 692/1403] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:06:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:06:56.424956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43788243-9f9c-46ba-a2ed-3800c1fab2d1 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:56.425539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43788243-9f9c-46ba-a2ed-3800c1fab2d1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:56.425539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43788243-9f9c-46ba-a2ed-3800c1fab2d1 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:56.427120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 712/1404] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:56.432150 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4539c-33cb-4b7b-8707-b47869a7f735 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:56.432614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4539c-33cb-4b7b-8707-b47869a7f735 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:56.432708 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e4539c-33cb-4b7b-8707-b47869a7f735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:56.514307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f4e4539c-33cb-4b7b-8707-b47869a7f735 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:56.514307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f4e4539c-33cb-4b7b-8707-b47869a7f735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:56.641443 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 693/1405] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:56.692901 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43788243-9f9c-46ba-a2ed-3800c1fab2d1 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:56.694557 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-43788243-9f9c-46ba-a2ed-3800c1fab2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:56.801290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:56.868381 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43788243-9f9c-46ba-a2ed-3800c1fab2d1 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:56.999880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381cc15-24bf-406a-b206-c76ef1fb6f77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:57.000453 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381cc15-24bf-406a-b206-c76ef1fb6f77 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:57.000735 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381cc15-24bf-406a-b206-c76ef1fb6f77 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:57.001041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381cc15-24bf-406a-b206-c76ef1fb6f77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:57.002695 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e381cc15-24bf-406a-b206-c76ef1fb6f77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:57.077960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:57.112323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381cc15-24bf-406a-b206-c76ef1fb6f77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:57.114077 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381cc15-24bf-406a-b206-c76ef1fb6f77 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:57.136306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 713/1406] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:06:59.268458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f0dc2-6599-4c8e-a163-1277d696a0f3 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:59.269057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f0dc2-6599-4c8e-a163-1277d696a0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:59.269057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8f0dc2-6599-4c8e-a163-1277d696a0f3 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:59.270626 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 694/1407] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:06:59.276835 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c23c8f-3f4f-438a-b21f-d72da7d12fa4 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:59.277328 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c23c8f-3f4f-438a-b21f-d72da7d12fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:59.277422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c23c8f-3f4f-438a-b21f-d72da7d12fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:59.347345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-13c23c8f-3f4f-438a-b21f-d72da7d12fa4 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:06:59.348576 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-13c23c8f-3f4f-438a-b21f-d72da7d12fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:59.458161 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 714/1408] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:06:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:06:59.502365 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f0dc2-6599-4c8e-a163-1277d696a0f3 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:06:59.503907 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-dd8f0dc2-6599-4c8e-a163-1277d696a0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:59.589820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:59.668442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8f0dc2-6599-4c8e-a163-1277d696a0f3 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:06:59.793796 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69050e33-39cb-4ff2-b5d2-7362adafcc43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:06:59.793796 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69050e33-39cb-4ff2-b5d2-7362adafcc43 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:06:59.793796 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69050e33-39cb-4ff2-b5d2-7362adafcc43 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:06:59.794179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69050e33-39cb-4ff2-b5d2-7362adafcc43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:06:59.795724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-69050e33-39cb-4ff2-b5d2-7362adafcc43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:06:59.835220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:06:59.900126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69050e33-39cb-4ff2-b5d2-7362adafcc43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:06:59.901776 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69050e33-39cb-4ff2-b5d2-7362adafcc43 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:06:59.926022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 695/1409] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:06:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:09.456118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=85 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:09.918814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=30 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:16.678089 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bf19f-59d6-4329-8c76-461e7f35c90a None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:16.678662 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bf19f-59d6-4329-8c76-461e7f35c90a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:16.678662 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495bf19f-59d6-4329-8c76-461e7f35c90a None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:16.679983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 715/1410] 10.222.0.61 () {54 vars in 918 bytes} [Thu Jul 14 10:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:16.684987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73a39b-00cf-4257-b7dd-4f288d0498ea None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:16.686006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73a39b-00cf-4257-b7dd-4f288d0498ea None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:16.686112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b73a39b-00cf-4257-b7dd-4f288d0498ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:16.759964 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6b73a39b-00cf-4257-b7dd-4f288d0498ea None None] MFA Rules not processed for user `ff4c42a4f4f549378e5bc1cd92f801c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:16.761132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6b73a39b-00cf-4257-b7dd-4f288d0498ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:16.869364 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 696/1411] 10.222.0.61 () {58 vars in 1015 bytes} [Thu Jul 14 10:07:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:16.883546 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835695b-418b-4749-ae0e-52020b2fa1cc None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:16.883828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835695b-418b-4749-ae0e-52020b2fa1cc None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:16.884104 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1835695b-418b-4749-ae0e-52020b2fa1cc None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:16.885505 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 716/1412] 10.222.0.61 () {54 vars in 918 bytes} [Thu Jul 14 10:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:16.891096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f512031-5c0e-4789-aea7-841f5ed0a4b2 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:16.891382 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f512031-5c0e-4789-aea7-841f5ed0a4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:16.891653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f512031-5c0e-4789-aea7-841f5ed0a4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:16.962260 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:16.965291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4f512031-5c0e-4789-aea7-841f5ed0a4b2 None None] MFA Rules not processed for user `e9f4efa3e2f343c2ac858a277601a7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:16.967709 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4f512031-5c0e-4789-aea7-841f5ed0a4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:17.083022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 697/1413] 10.222.0.61 () {58 vars in 1015 bytes} [Thu Jul 14 10:07:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:19.538676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d901a-7913-493f-8037-5ae4f8dddb81 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:19.539235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d901a-7913-493f-8037-5ae4f8dddb81 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:19.539235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d901a-7913-493f-8037-5ae4f8dddb81 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:19.540560 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:19.546480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12692feb-4cf4-4be7-8e78-22a68b02599f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:19.546932 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12692feb-4cf4-4be7-8e78-22a68b02599f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:19.547038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12692feb-4cf4-4be7-8e78-22a68b02599f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:19.610704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-12692feb-4cf4-4be7-8e78-22a68b02599f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:19.611915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-12692feb-4cf4-4be7-8e78-22a68b02599f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:19.666012 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:19.710168 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:07:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:19.752658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d901a-7913-493f-8037-5ae4f8dddb81 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:19.754243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-385d901a-7913-493f-8037-5ae4f8dddb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:19.896961 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d901a-7913-493f-8037-5ae4f8dddb81 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:19.967280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:20.006039 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36357cde-125c-4420-8abc-d056d65f2bb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:20.006674 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36357cde-125c-4420-8abc-d056d65f2bb5 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:20.006952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36357cde-125c-4420-8abc-d056d65f2bb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:20.007219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36357cde-125c-4420-8abc-d056d65f2bb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:20.008848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-36357cde-125c-4420-8abc-d056d65f2bb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:20.097588 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36357cde-125c-4420-8abc-d056d65f2bb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:20.099176 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36357cde-125c-4420-8abc-d056d65f2bb5 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:20.116656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 718/1416] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:21.013769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacef841-aa68-4829-958a-6900bae807f8 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:21.014334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacef841-aa68-4829-958a-6900bae807f8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:21.014334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacef841-aa68-4829-958a-6900bae807f8 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:21.015705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:21.020745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6215f41-6494-4086-8660-4a91f1050c92 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:21.021253 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6215f41-6494-4086-8660-4a91f1050c92 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:21.021539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6215f41-6494-4086-8660-4a91f1050c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:21.094590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a6215f41-6494-4086-8660-4a91f1050c92 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:21.095781 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a6215f41-6494-4086-8660-4a91f1050c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:21.121749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:21.309039 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:07:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:21.461000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacef841-aa68-4829-958a-6900bae807f8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:21.462513 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cacef841-aa68-4829-958a-6900bae807f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:21.606800 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacef841-aa68-4829-958a-6900bae807f8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:21.646766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:21.720887 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e11761-c8d4-4e54-b193-ec9fd7d21c8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:21.721566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e11761-c8d4-4e54-b193-ec9fd7d21c8e service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:21.721847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e11761-c8d4-4e54-b193-ec9fd7d21c8e service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:21.722118 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e11761-c8d4-4e54-b193-ec9fd7d21c8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:21.723779 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-16e11761-c8d4-4e54-b193-ec9fd7d21c8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:21.814171 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e11761-c8d4-4e54-b193-ec9fd7d21c8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:21.815805 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e11761-c8d4-4e54-b193-ec9fd7d21c8e service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:21.833832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 700/1419] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:22.721027 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae8d4c-4236-478d-be8a-dd360162b080 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:22.721602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae8d4c-4236-478d-be8a-dd360162b080 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:22.721602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae8d4c-4236-478d-be8a-dd360162b080 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:22.722958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:07:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:22.727897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cac10-c159-49e7-9261-90cf5fdc7c27 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:22.728283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cac10-c159-49e7-9261-90cf5fdc7c27 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:22.728440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cac10-c159-49e7-9261-90cf5fdc7c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:22.781368 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:22.804128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7b1cac10-c159-49e7-9261-90cf5fdc7c27 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:22.805405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7b1cac10-c159-49e7-9261-90cf5fdc7c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:22.900356 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:23.054654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae8d4c-4236-478d-be8a-dd360162b080 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:23.056082 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a1ae8d4c-4236-478d-be8a-dd360162b080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:23.201247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ae8d4c-4236-478d-be8a-dd360162b080 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:23.213000 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:23.312741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a0b5e8-5487-486a-8c71-e4dbdde021af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:23.313412 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0b5e8-5487-486a-8c71-e4dbdde021af service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:23.313685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0b5e8-5487-486a-8c71-e4dbdde021af service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:23.313952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a0b5e8-5487-486a-8c71-e4dbdde021af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:23.315594 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f0a0b5e8-5487-486a-8c71-e4dbdde021af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:23.401273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0b5e8-5487-486a-8c71-e4dbdde021af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:23.402847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a0b5e8-5487-486a-8c71-e4dbdde021af service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:23.421093 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 721/1422] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:24.280220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65b7c5-bd57-4812-966a-9a796893e392 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:24.280671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65b7c5-bd57-4812-966a-9a796893e392 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:24.280976 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a65b7c5-bd57-4812-966a-9a796893e392 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:24.282186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:07:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:24.287347 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0c64c-4c5e-45af-a594-ed167e73cdef None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:24.287761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0c64c-4c5e-45af-a594-ed167e73cdef None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:24.287877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de0c64c-4c5e-45af-a594-ed167e73cdef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:24.305731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:24.361177 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6de0c64c-4c5e-45af-a594-ed167e73cdef None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:24.362365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6de0c64c-4c5e-45af-a594-ed167e73cdef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:24.455996 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:07:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:24.605861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65b7c5-bd57-4812-966a-9a796893e392 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:24.607393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8a65b7c5-bd57-4812-966a-9a796893e392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:24.714206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:24.751928 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a65b7c5-bd57-4812-966a-9a796893e392 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:24.861704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153801b5-6eb0-4b69-a17a-c3baa9eaa3f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:24.862354 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153801b5-6eb0-4b69-a17a-c3baa9eaa3f7 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:24.862625 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153801b5-6eb0-4b69-a17a-c3baa9eaa3f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:24.862893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153801b5-6eb0-4b69-a17a-c3baa9eaa3f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:24.864524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-153801b5-6eb0-4b69-a17a-c3baa9eaa3f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:24.939315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:24.952805 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153801b5-6eb0-4b69-a17a-c3baa9eaa3f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:24.954517 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153801b5-6eb0-4b69-a17a-c3baa9eaa3f7 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:24.971999 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 703/1425] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:25.857752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46f2d9-5486-4a5a-8457-0bd356f7f62f None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:25.858322 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46f2d9-5486-4a5a-8457-0bd356f7f62f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:25.858322 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a46f2d9-5486-4a5a-8457-0bd356f7f62f None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:25.859656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:07:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:25.864506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05679211-79a4-4c65-9e5c-89bc9dc31376 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:25.864960 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05679211-79a4-4c65-9e5c-89bc9dc31376 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:25.865074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05679211-79a4-4c65-9e5c-89bc9dc31376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:25.939382 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-05679211-79a4-4c65-9e5c-89bc9dc31376 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:25.940582 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-05679211-79a4-4c65-9e5c-89bc9dc31376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:26.035253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:07:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:26.196424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46f2d9-5486-4a5a-8457-0bd356f7f62f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:26.197983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4a46f2d9-5486-4a5a-8457-0bd356f7f62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:26.282772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:26.343862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a46f2d9-5486-4a5a-8457-0bd356f7f62f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:26.452899 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f63f28e-1e09-45ce-8505-629f994f015f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:26.453558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63f28e-1e09-45ce-8505-629f994f015f service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:26.453833 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63f28e-1e09-45ce-8505-629f994f015f service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:26.454100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f63f28e-1e09-45ce-8505-629f994f015f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:26.455728 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8f63f28e-1e09-45ce-8505-629f994f015f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:26.502544 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:26.543200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63f28e-1e09-45ce-8505-629f994f015f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:26.545323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f63f28e-1e09-45ce-8505-629f994f015f service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:26.563598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 724/1428] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:27.377379 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4260c-d6da-43e5-8201-3e8d082aff64 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:27.377903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4260c-d6da-43e5-8201-3e8d082aff64 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:27.377994 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4260c-d6da-43e5-8201-3e8d082aff64 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:27.379328 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:27.384352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0565822-3f03-4ab7-bb59-58bc163bdeb3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:27.384775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0565822-3f03-4ab7-bb59-58bc163bdeb3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:27.384904 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0565822-3f03-4ab7-bb59-58bc163bdeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:27.459197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d0565822-3f03-4ab7-bb59-58bc163bdeb3 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:27.460394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d0565822-3f03-4ab7-bb59-58bc163bdeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:27.553832 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:27.702786 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4260c-d6da-43e5-8201-3e8d082aff64 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:27.704371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-22a4260c-d6da-43e5-8201-3e8d082aff64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:27.754063 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:27.853087 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4260c-d6da-43e5-8201-3e8d082aff64 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:27.964836 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bf2f4-f9cd-499a-878d-6a5afb2710f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:27.965525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bf2f4-f9cd-499a-878d-6a5afb2710f6 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:27.965809 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bf2f4-f9cd-499a-878d-6a5afb2710f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:27.966637 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bf2f4-f9cd-499a-878d-6a5afb2710f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:27.968275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a42bf2f4-f9cd-499a-878d-6a5afb2710f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:27.975070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:28.058411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42bf2f4-f9cd-499a-878d-6a5afb2710f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:28.060661 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42bf2f4-f9cd-499a-878d-6a5afb2710f6 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:28.078174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 706/1431] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:28.885879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5390357-15fa-4b43-a02f-aff7193ff555 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:28.886474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5390357-15fa-4b43-a02f-aff7193ff555 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:28.886474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5390357-15fa-4b43-a02f-aff7193ff555 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:28.888290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:07:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:28.893292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e735ab8-dc51-4f08-a11a-c8eb29e0c612 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:28.893746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e735ab8-dc51-4f08-a11a-c8eb29e0c612 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:28.893865 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e735ab8-dc51-4f08-a11a-c8eb29e0c612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:28.969766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0e735ab8-dc51-4f08-a11a-c8eb29e0c612 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:28.970974 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0e735ab8-dc51-4f08-a11a-c8eb29e0c612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:29.038331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:29.068266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:07:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:29.221848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5390357-15fa-4b43-a02f-aff7193ff555 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:29.223381 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e5390357-15fa-4b43-a02f-aff7193ff555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:29.241997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:29.367850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5390357-15fa-4b43-a02f-aff7193ff555 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:29.452228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:29.478978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fdecea-5ec5-4b66-a6bd-b5a5c7c8b583 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:29.479629 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdecea-5ec5-4b66-a6bd-b5a5c7c8b583 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:29.479908 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdecea-5ec5-4b66-a6bd-b5a5c7c8b583 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:29.480179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fdecea-5ec5-4b66-a6bd-b5a5c7c8b583 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:29.481884 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b2fdecea-5ec5-4b66-a6bd-b5a5c7c8b583 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:29.568767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdecea-5ec5-4b66-a6bd-b5a5c7c8b583 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:29.570397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fdecea-5ec5-4b66-a6bd-b5a5c7c8b583 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:29.588183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 727/1434] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:30.399170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79725c-29cd-4287-af23-6c10631a35d7 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:30.399740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79725c-29cd-4287-af23-6c10631a35d7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:30.399740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79725c-29cd-4287-af23-6c10631a35d7 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:30.401142 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:30.406068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6c46a-6548-48df-ba07-4ef220d50905 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:30.406489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6c46a-6548-48df-ba07-4ef220d50905 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:30.406625 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e6c46a-6548-48df-ba07-4ef220d50905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:30.480752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d2e6c46a-6548-48df-ba07-4ef220d50905 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:30.482425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d2e6c46a-6548-48df-ba07-4ef220d50905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:30.522698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:30.577345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:30.729949 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79725c-29cd-4287-af23-6c10631a35d7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:30.732094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7a79725c-29cd-4287-af23-6c10631a35d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:30.886888 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79725c-29cd-4287-af23-6c10631a35d7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:30.942128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:31.002319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddef0a1-801d-4da2-93cd-17f3902b2a6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:31.002891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddef0a1-801d-4da2-93cd-17f3902b2a6c service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:31.003167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddef0a1-801d-4da2-93cd-17f3902b2a6c service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:31.003437 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddef0a1-801d-4da2-93cd-17f3902b2a6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:31.005108 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5ddef0a1-801d-4da2-93cd-17f3902b2a6c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:31.095197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddef0a1-801d-4da2-93cd-17f3902b2a6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:31.096802 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddef0a1-801d-4da2-93cd-17f3902b2a6c service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:31.114497 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 709/1437] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:31.920293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8737126-c0ca-4539-a10c-c0ad95b13759 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:31.920856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8737126-c0ca-4539-a10c-c0ad95b13759 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:31.920856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8737126-c0ca-4539-a10c-c0ad95b13759 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:31.922261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:07:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:07:31.927153 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7da983-e182-4dd9-9627-6aa960f0639f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:31.927584 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7da983-e182-4dd9-9627-6aa960f0639f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:31.927690 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7da983-e182-4dd9-9627-6aa960f0639f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:32.002128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-aa7da983-e182-4dd9-9627-6aa960f0639f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:32.003327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-aa7da983-e182-4dd9-9627-6aa960f0639f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:32.005343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:32.104683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:32.256087 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8737126-c0ca-4539-a10c-c0ad95b13759 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:32.257621 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c8737126-c0ca-4539-a10c-c0ad95b13759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:32.402799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8737126-c0ca-4539-a10c-c0ad95b13759 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:32.429035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:32.513985 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7bbe34-5707-4bdc-9f2d-69b8f70e9ca6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:32.514629 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bbe34-5707-4bdc-9f2d-69b8f70e9ca6 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:32.514906 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bbe34-5707-4bdc-9f2d-69b8f70e9ca6 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:32.515175 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7bbe34-5707-4bdc-9f2d-69b8f70e9ca6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:32.516818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-db7bbe34-5707-4bdc-9f2d-69b8f70e9ca6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:32.603137 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7bbe34-5707-4bdc-9f2d-69b8f70e9ca6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:32.604713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7bbe34-5707-4bdc-9f2d-69b8f70e9ca6 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:32.623447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 730/1440] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:34.535489 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a6a1b-9a54-484f-bb02-a809937fcdb8 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:34.536302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a6a1b-9a54-484f-bb02-a809937fcdb8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:34.536302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0a6a1b-9a54-484f-bb02-a809937fcdb8 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:34.537616 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:34.542772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a2033-00a4-4bca-9b33-6f4ca93a654e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:34.543170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a2033-00a4-4bca-9b33-6f4ca93a654e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:34.543360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a2033-00a4-4bca-9b33-6f4ca93a654e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:34.583606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:34.610666 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-208a2033-00a4-4bca-9b33-6f4ca93a654e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:34.611909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-208a2033-00a4-4bca-9b33-6f4ca93a654e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:34.706654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:34.744476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a6a1b-9a54-484f-bb02-a809937fcdb8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:34.746044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8e0a6a1b-9a54-484f-bb02-a809937fcdb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:34.848739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0a6a1b-9a54-484f-bb02-a809937fcdb8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:34.882991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:34.924422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ebfdc-54b3-4c3f-b026-73d5eaa8c65c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:34.925113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ebfdc-54b3-4c3f-b026-73d5eaa8c65c service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:34.925395 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ebfdc-54b3-4c3f-b026-73d5eaa8c65c service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:34.925665 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ebfdc-54b3-4c3f-b026-73d5eaa8c65c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:34.927303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-067ebfdc-54b3-4c3f-b026-73d5eaa8c65c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:35.039466 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ebfdc-54b3-4c3f-b026-73d5eaa8c65c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:35.040891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ebfdc-54b3-4c3f-b026-73d5eaa8c65c service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:35.059012 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 712/1443] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:37.287954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad062f9-ea26-4ce3-a05c-212d62dab1e7 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:37.288450 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad062f9-ea26-4ce3-a05c-212d62dab1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:37.288582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad062f9-ea26-4ce3-a05c-212d62dab1e7 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:37.289873 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:37.295136 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7084324-c85b-4b09-bc28-d9ebea5ab62d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:37.295580 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7084324-c85b-4b09-bc28-d9ebea5ab62d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:37.295727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7084324-c85b-4b09-bc28-d9ebea5ab62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:37.360288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e7084324-c85b-4b09-bc28-d9ebea5ab62d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:37.361513 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e7084324-c85b-4b09-bc28-d9ebea5ab62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:37.374659 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:37.459999 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:37.497627 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad062f9-ea26-4ce3-a05c-212d62dab1e7 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:37.499204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fad062f9-ea26-4ce3-a05c-212d62dab1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:37.601241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad062f9-ea26-4ce3-a05c-212d62dab1e7 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:37.613480 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:37.674094 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65e17e-f449-4111-a8ed-8e19494baca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:37.674739 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65e17e-f449-4111-a8ed-8e19494baca8 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:37.675020 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65e17e-f449-4111-a8ed-8e19494baca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:37.675298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65e17e-f449-4111-a8ed-8e19494baca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:37.676970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6b65e17e-f449-4111-a8ed-8e19494baca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:37.783383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65e17e-f449-4111-a8ed-8e19494baca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:37.784797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65e17e-f449-4111-a8ed-8e19494baca8 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:37.803452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 733/1446] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:40.160336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6b3bd-9d0d-45a5-a5a6-62cc670c3b9b None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:40.161194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6b3bd-9d0d-45a5-a5a6-62cc670c3b9b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:40.161194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6b3bd-9d0d-45a5-a5a6-62cc670c3b9b None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:40.162581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:40.167770 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebcfd7-c939-4154-b3f7-012f71e725dd None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:40.168706 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebcfd7-c939-4154-b3f7-012f71e725dd None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:40.168846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ebcfd7-c939-4154-b3f7-012f71e725dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:40.214646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:40.232466 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f5ebcfd7-c939-4154-b3f7-012f71e725dd None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:40.233699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f5ebcfd7-c939-4154-b3f7-012f71e725dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:40.330396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:40.370045 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6b3bd-9d0d-45a5-a5a6-62cc670c3b9b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:40.371589 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6bf6b3bd-9d0d-45a5-a5a6-62cc670c3b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:40.473931 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6b3bd-9d0d-45a5-a5a6-62cc670c3b9b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:40.513885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:40.549460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7e930-f07b-4cf8-b9d0-733ca615c00d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:40.550094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7e930-f07b-4cf8-b9d0-733ca615c00d service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:40.550376 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7e930-f07b-4cf8-b9d0-733ca615c00d service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:40.550653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7e930-f07b-4cf8-b9d0-733ca615c00d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:40.552294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-edf7e930-f07b-4cf8-b9d0-733ca615c00d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:40.662085 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7e930-f07b-4cf8-b9d0-733ca615c00d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:40.663508 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7e930-f07b-4cf8-b9d0-733ca615c00d service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:40.681390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 715/1449] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:42.818844 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54ed0d-fd32-44bf-bd3d-2ddfa8be908f None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:42.819406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54ed0d-fd32-44bf-bd3d-2ddfa8be908f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:42.819406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db54ed0d-fd32-44bf-bd3d-2ddfa8be908f None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:42.820772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:42.826410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5a92c-e0e8-4dcb-b424-239c436cf9b5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:42.826847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5a92c-e0e8-4dcb-b424-239c436cf9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:42.826972 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5a92c-e0e8-4dcb-b424-239c436cf9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:42.890374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-dbd5a92c-e0e8-4dcb-b424-239c436cf9b5 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:42.891588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-dbd5a92c-e0e8-4dcb-b424-239c436cf9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:42.912144 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:42.990929 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:43.029664 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54ed0d-fd32-44bf-bd3d-2ddfa8be908f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:43.031331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-db54ed0d-fd32-44bf-bd3d-2ddfa8be908f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:43.132267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db54ed0d-fd32-44bf-bd3d-2ddfa8be908f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:43.151072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:43.205051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d66db-b149-4fb5-8c2b-6d46deecceb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:43.205701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d66db-b149-4fb5-8c2b-6d46deecceb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:43.205981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d66db-b149-4fb5-8c2b-6d46deecceb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:43.206255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d66db-b149-4fb5-8c2b-6d46deecceb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:43.207902 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b79d66db-b149-4fb5-8c2b-6d46deecceb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:43.315657 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d66db-b149-4fb5-8c2b-6d46deecceb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:43.317193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d66db-b149-4fb5-8c2b-6d46deecceb2 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:43.335407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 736/1452] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:45.433163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cab4de-466b-4efa-ad08-d1173da5b60d None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:45.433707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cab4de-466b-4efa-ad08-d1173da5b60d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:45.433825 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cab4de-466b-4efa-ad08-d1173da5b60d None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:45.435171 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:07:45.440560 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2d997-7121-48db-a909-6018d6977633 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:45.441014 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2d997-7121-48db-a909-6018d6977633 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:45.441126 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2d997-7121-48db-a909-6018d6977633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:45.501883 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:45.505333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-55c2d997-7121-48db-a909-6018d6977633 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:45.506817 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-55c2d997-7121-48db-a909-6018d6977633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:45.602391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:45.640154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cab4de-466b-4efa-ad08-d1173da5b60d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:45.641711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c3cab4de-466b-4efa-ad08-d1173da5b60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:45.744187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cab4de-466b-4efa-ad08-d1173da5b60d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:45.793011 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:45.817611 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c4bd7e-197e-472c-9dda-4a5e47c88e7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:45.818259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4bd7e-197e-472c-9dda-4a5e47c88e7c service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:45.818543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4bd7e-197e-472c-9dda-4a5e47c88e7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:45.818819 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c4bd7e-197e-472c-9dda-4a5e47c88e7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:45.820453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-68c4bd7e-197e-472c-9dda-4a5e47c88e7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:45.928937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4bd7e-197e-472c-9dda-4a5e47c88e7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:45.930408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c4bd7e-197e-472c-9dda-4a5e47c88e7c service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:45.950318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 718/1455] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:48.073299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330643c0-dde9-4be8-b392-0c0d6ec8889e None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:48.074173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330643c0-dde9-4be8-b392-0c0d6ec8889e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:48.074420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330643c0-dde9-4be8-b392-0c0d6ec8889e None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:48.076527 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:07:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:48.081692 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c6324-e546-4f9a-9de5-c692298c5804 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:48.082086 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c6324-e546-4f9a-9de5-c692298c5804 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:48.082232 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97c6324-e546-4f9a-9de5-c692298c5804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:48.263067 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a97c6324-e546-4f9a-9de5-c692298c5804 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:48.264276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a97c6324-e546-4f9a-9de5-c692298c5804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:48.292663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:48.360026 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:48.397309 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330643c0-dde9-4be8-b392-0c0d6ec8889e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:48.398833 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-330643c0-dde9-4be8-b392-0c0d6ec8889e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:48.501016 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330643c0-dde9-4be8-b392-0c0d6ec8889e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:48.528893 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:48.573856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d588910-4469-4852-be28-f4d69c36edfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:48.574502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d588910-4469-4852-be28-f4d69c36edfd service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:48.574785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d588910-4469-4852-be28-f4d69c36edfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:48.575061 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d588910-4469-4852-be28-f4d69c36edfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:48.576804 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8d588910-4469-4852-be28-f4d69c36edfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:48.684657 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d588910-4469-4852-be28-f4d69c36edfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:48.686126 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d588910-4469-4852-be28-f4d69c36edfd service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:48.704080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 739/1458] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:50.754604 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba3e62-81e3-4c8c-ad7c-eb32b658155b None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:50.755145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba3e62-81e3-4c8c-ad7c-eb32b658155b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:50.755145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba3e62-81e3-4c8c-ad7c-eb32b658155b None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:50.756531 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:07:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:50.761839 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24648324-9d23-4696-b7e0-6e5a0f2aeeb6 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:50.762242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24648324-9d23-4696-b7e0-6e5a0f2aeeb6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:50.762390 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24648324-9d23-4696-b7e0-6e5a0f2aeeb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:50.825162 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-24648324-9d23-4696-b7e0-6e5a0f2aeeb6 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:50.826345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-24648324-9d23-4696-b7e0-6e5a0f2aeeb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:50.834449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:50.921477 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:07:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:07:50.959428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba3e62-81e3-4c8c-ad7c-eb32b658155b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:50.961008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-63ba3e62-81e3-4c8c-ad7c-eb32b658155b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:51.063884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba3e62-81e3-4c8c-ad7c-eb32b658155b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:51.123830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:51.142517 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17584622-c2d0-41bb-a3f0-8d64e57d3579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:51.143157 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17584622-c2d0-41bb-a3f0-8d64e57d3579 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:51.143438 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17584622-c2d0-41bb-a3f0-8d64e57d3579 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:51.143712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17584622-c2d0-41bb-a3f0-8d64e57d3579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:51.145368 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-17584622-c2d0-41bb-a3f0-8d64e57d3579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:51.252416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17584622-c2d0-41bb-a3f0-8d64e57d3579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:51.253869 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17584622-c2d0-41bb-a3f0-8d64e57d3579 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:51.271334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 721/1461] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:53.328037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d111db-2a6f-49fb-9821-1ce96bb2711f None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:53.328551 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d111db-2a6f-49fb-9821-1ce96bb2711f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:53.328671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d111db-2a6f-49fb-9821-1ce96bb2711f None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:53.329974 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:53.335409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb4057-609b-483a-8907-82869c694243 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:53.335795 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb4057-609b-483a-8907-82869c694243 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:53.335955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fb4057-609b-483a-8907-82869c694243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:53.398595 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-67fb4057-609b-483a-8907-82869c694243 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:53.399789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-67fb4057-609b-483a-8907-82869c694243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:53.439360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:53.497097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:53.533010 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d111db-2a6f-49fb-9821-1ce96bb2711f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:53.534568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f1d111db-2a6f-49fb-9821-1ce96bb2711f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:53.635485 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d111db-2a6f-49fb-9821-1ce96bb2711f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:53.671559 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:53.705900 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe87210-1ad1-421b-a459-eb20cb1d5ef3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:53.706534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe87210-1ad1-421b-a459-eb20cb1d5ef3 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:53.706812 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe87210-1ad1-421b-a459-eb20cb1d5ef3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:53.707086 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe87210-1ad1-421b-a459-eb20cb1d5ef3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:53.708713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-abe87210-1ad1-421b-a459-eb20cb1d5ef3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:53.816176 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe87210-1ad1-421b-a459-eb20cb1d5ef3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:53.817574 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe87210-1ad1-421b-a459-eb20cb1d5ef3 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:53.835421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 742/1464] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:55.843691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d270f-5e89-4eee-ba7d-2521a0d87fbc None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:55.844353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d270f-5e89-4eee-ba7d-2521a0d87fbc None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:55.844353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d270f-5e89-4eee-ba7d-2521a0d87fbc None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:55.845685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:55.850869 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c69a8-331f-49ec-be30-48ccc8b011f1 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:55.851276 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c69a8-331f-49ec-be30-48ccc8b011f1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:55.851416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2c69a8-331f-49ec-be30-48ccc8b011f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:55.914001 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-fe2c69a8-331f-49ec-be30-48ccc8b011f1 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:55.915194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fe2c69a8-331f-49ec-be30-48ccc8b011f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:55.932346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:56.008385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:56.044142 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d270f-5e89-4eee-ba7d-2521a0d87fbc None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:56.045685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-890d270f-5e89-4eee-ba7d-2521a0d87fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:56.146814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d270f-5e89-4eee-ba7d-2521a0d87fbc None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:56.211797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:56.218902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8099f-3e9e-4193-b259-17e8cce8a8e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:56.219543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8099f-3e9e-4193-b259-17e8cce8a8e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:56.219816 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8099f-3e9e-4193-b259-17e8cce8a8e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:56.220082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8099f-3e9e-4193-b259-17e8cce8a8e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:56.221763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-58d8099f-3e9e-4193-b259-17e8cce8a8e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:56.332021 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8099f-3e9e-4193-b259-17e8cce8a8e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:56.333483 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8099f-3e9e-4193-b259-17e8cce8a8e3 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:56.351870 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 724/1467] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:07:58.369046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cea32-1834-4a21-8c20-2b6c0b48645c None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:58.369608 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cea32-1834-4a21-8c20-2b6c0b48645c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:58.369608 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024cea32-1834-4a21-8c20-2b6c0b48645c None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:58.371566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:07:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:07:58.376936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27995307-dc73-4590-a534-77c2652057cb None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:58.377351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27995307-dc73-4590-a534-77c2652057cb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:58.377499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27995307-dc73-4590-a534-77c2652057cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:58.442189 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-27995307-dc73-4590-a534-77c2652057cb None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:07:58.443311 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-27995307-dc73-4590-a534-77c2652057cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:58.492065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:58.540386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:07:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:07:58.576004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cea32-1834-4a21-8c20-2b6c0b48645c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:07:58.577604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-024cea32-1834-4a21-8c20-2b6c0b48645c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:58.682057 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024cea32-1834-4a21-8c20-2b6c0b48645c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:07:58.728776 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:07:58.755699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a70b1-a08f-45cd-a8fa-edc4fcb3ba7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:07:58.756351 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a70b1-a08f-45cd-a8fa-edc4fcb3ba7b service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:07:58.756631 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a70b1-a08f-45cd-a8fa-edc4fcb3ba7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:07:58.756907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a70b1-a08f-45cd-a8fa-edc4fcb3ba7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:07:58.758646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-487a70b1-a08f-45cd-a8fa-edc4fcb3ba7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:07:58.870383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a70b1-a08f-45cd-a8fa-edc4fcb3ba7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:07:58.872345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a70b1-a08f-45cd-a8fa-edc4fcb3ba7b service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:07:58.890584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 745/1470] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:07:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:08:08.538640 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=22 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:08.888057 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=66 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:18.510940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba7c96-33a9-408d-baea-37b59e77114c None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:18.511500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba7c96-33a9-408d-baea-37b59e77114c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:18.511500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ba7c96-33a9-408d-baea-37b59e77114c None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:18.512782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 726/1471] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:08:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:08:18.517993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f432ee4-1dc6-41ea-9610-1710b0c81f94 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:18.518438 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f432ee4-1dc6-41ea-9610-1710b0c81f94 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:18.518542 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f432ee4-1dc6-41ea-9610-1710b0c81f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:18.582477 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1f432ee4-1dc6-41ea-9610-1710b0c81f94 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:08:18.583686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1f432ee4-1dc6-41ea-9610-1710b0c81f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:18.677394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 746/1472] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:08:18.761390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e2053-ddad-4fd0-af77-133f048cd46f None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:18.761626 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e2053-ddad-4fd0-af77-133f048cd46f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:18.761931 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e2053-ddad-4fd0-af77-133f048cd46f None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:18.763304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 727/1473] 10.222.0.61 () {54 vars in 962 bytes} [Thu Jul 14 10:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:08:18.768130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992168c-f6e9-4c68-a652-4dab46f26859 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:18.768420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992168c-f6e9-4c68-a652-4dab46f26859 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:18.768699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b992168c-f6e9-4c68-a652-4dab46f26859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:18.846740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-b992168c-f6e9-4c68-a652-4dab46f26859 None None] MFA Rules not processed for user `e3b5660cfd344dcd8abe14c88df686e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:08:18.847941 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b992168c-f6e9-4c68-a652-4dab46f26859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:18.867487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:18.923048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 747/1474] 10.222.0.61 () {58 vars in 1059 bytes} [Thu Jul 14 10:08:18 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:08:18.930509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e2053-ddad-4fd0-af77-133f048cd46f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:08:18.932612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ed4e2053-ddad-4fd0-af77-133f048cd46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:19.033106 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e2053-ddad-4fd0-af77-133f048cd46f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:08:19.104428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adb8fde-15cb-4508-b44e-4694d831f861 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b5660cfd344dcd8abe14c88df686e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:08:19.105095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb8fde-15cb-4508-b44e-4694d831f861 service glance] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:19.105372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb8fde-15cb-4508-b44e-4694d831f861 service glance] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:19.105636 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adb8fde-15cb-4508-b44e-4694d831f861 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:19.107256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6adb8fde-15cb-4508-b44e-4694d831f861 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:19.160777 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:19.215098 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb8fde-15cb-4508-b44e-4694d831f861 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:08:19.216526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adb8fde-15cb-4508-b44e-4694d831f861 service glance] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:08:19.234687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 728/1475] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:08:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:08:21.322702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fc43a-b820-4294-88d4-5d321a68106e None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:21.323249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fc43a-b820-4294-88d4-5d321a68106e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:21.323249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fc43a-b820-4294-88d4-5d321a68106e None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:21.324617 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 748/1476] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:08:21.329692 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60f650-c15c-4a5e-a0cf-3d0299dcd986 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:21.330116 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60f650-c15c-4a5e-a0cf-3d0299dcd986 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:21.330232 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e60f650-c15c-4a5e-a0cf-3d0299dcd986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:21.394850 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8e60f650-c15c-4a5e-a0cf-3d0299dcd986 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:08:21.396049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8e60f650-c15c-4a5e-a0cf-3d0299dcd986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:21.493022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 729/1477] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:08:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:08:21.516847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fc43a-b820-4294-88d4-5d321a68106e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:08:21.518393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fb4fc43a-b820-4294-88d4-5d321a68106e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:21.641087 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fc43a-b820-4294-88d4-5d321a68106e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:08:21.688014 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:21.731682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d619c63-9a7e-4743-8def-1dcde6d33b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:08:21.732328 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d619c63-9a7e-4743-8def-1dcde6d33b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:21.732603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d619c63-9a7e-4743-8def-1dcde6d33b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:21.732871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d619c63-9a7e-4743-8def-1dcde6d33b9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:21.733627 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d619c63-9a7e-4743-8def-1dcde6d33b9f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:08:21.735331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-5d619c63-9a7e-4743-8def-1dcde6d33b9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:08:21.735331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Jul 14 10:08:21.735331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: Jul 14 10:08:21.736450 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d619c63-9a7e-4743-8def-1dcde6d33b9f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:08:21.754844 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 749/1478] 10.222.0.61 () {56 vars in 1108 bytes} [Thu Jul 14 10:08:21 2022] GET /identity/v3/projects => generated 2553 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:08:23.815138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10644ce4-02e6-47ad-a81e-a5f8c9484e8a None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:23.815698 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10644ce4-02e6-47ad-a81e-a5f8c9484e8a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:23.815698 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10644ce4-02e6-47ad-a81e-a5f8c9484e8a None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:23.817106 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 730/1479] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:08:23.822828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf8fe3-098a-4330-bf08-33b73958ffba None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:23.823274 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf8fe3-098a-4330-bf08-33b73958ffba None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:23.823372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bf8fe3-098a-4330-bf08-33b73958ffba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:23.885857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-67bf8fe3-098a-4330-bf08-33b73958ffba None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:08:23.887051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-67bf8fe3-098a-4330-bf08-33b73958ffba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:23.981700 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 750/1480] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:08:24.024615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10644ce4-02e6-47ad-a81e-a5f8c9484e8a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:08:24.026942 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-10644ce4-02e6-47ad-a81e-a5f8c9484e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:24.036800 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:24.172034 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10644ce4-02e6-47ad-a81e-a5f8c9484e8a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:08:24.251351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:24.284583 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9a2954-c67d-47b0-8a67-cf2680abed78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:08:24.285263 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a2954-c67d-47b0-8a67-cf2680abed78 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:24.285547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a2954-c67d-47b0-8a67-cf2680abed78 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:24.285820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9a2954-c67d-47b0-8a67-cf2680abed78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:24.287456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fa9a2954-c67d-47b0-8a67-cf2680abed78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:24.375740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a2954-c67d-47b0-8a67-cf2680abed78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:08:24.377355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9a2954-c67d-47b0-8a67-cf2680abed78 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:08:24.395102 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 731/1481] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:08:26.610951 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121808f8-36ef-4aba-89c2-73b92b66d8c8 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:26.611483 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121808f8-36ef-4aba-89c2-73b92b66d8c8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:26.611483 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121808f8-36ef-4aba-89c2-73b92b66d8c8 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:26.612785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 751/1482] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:08:26.618073 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c1835-dc07-4b39-b32c-b052fa783393 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:26.618508 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c1835-dc07-4b39-b32c-b052fa783393 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:26.619158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c1835-dc07-4b39-b32c-b052fa783393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:26.683228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6c7c1835-dc07-4b39-b32c-b052fa783393 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:08:26.684416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6c7c1835-dc07-4b39-b32c-b052fa783393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:26.731422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:26.782331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 732/1483] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:08:26.820680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121808f8-36ef-4aba-89c2-73b92b66d8c8 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:08:26.822336 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-121808f8-36ef-4aba-89c2-73b92b66d8c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:26.846528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:26.926199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121808f8-36ef-4aba-89c2-73b92b66d8c8 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:08:26.998736 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12017a5c-03da-439c-ab35-24fb4a8d8c17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:08:26.999380 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12017a5c-03da-439c-ab35-24fb4a8d8c17 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:26.999658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12017a5c-03da-439c-ab35-24fb4a8d8c17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:26.999930 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12017a5c-03da-439c-ab35-24fb4a8d8c17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:27.001609 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-12017a5c-03da-439c-ab35-24fb4a8d8c17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:27.074501 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:27.109155 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12017a5c-03da-439c-ab35-24fb4a8d8c17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:08:27.110583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12017a5c-03da-439c-ab35-24fb4a8d8c17 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:08:27.129291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 752/1484] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:08:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:08:29.209878 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c87041-74a7-46aa-82a3-5b96c1adbf9d None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:29.210425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c87041-74a7-46aa-82a3-5b96c1adbf9d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:29.210425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c87041-74a7-46aa-82a3-5b96c1adbf9d None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:29.211803 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 733/1485] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:08:29.217601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20a38a-65c3-4860-b473-abc33d854466 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:29.217986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20a38a-65c3-4860-b473-abc33d854466 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:29.218167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20a38a-65c3-4860-b473-abc33d854466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:29.285490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-bb20a38a-65c3-4860-b473-abc33d854466 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:08:29.286700 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bb20a38a-65c3-4860-b473-abc33d854466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:29.380300 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 753/1486] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:08:29.417101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c87041-74a7-46aa-82a3-5b96c1adbf9d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:08:29.418635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d7c87041-74a7-46aa-82a3-5b96c1adbf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:29.521197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c87041-74a7-46aa-82a3-5b96c1adbf9d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:08:29.592743 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:29.594854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a82023-661a-4c16-899c-b3b37bca0a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:08:29.595615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a82023-661a-4c16-899c-b3b37bca0a98 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:29.595926 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a82023-661a-4c16-899c-b3b37bca0a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:29.596210 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a82023-661a-4c16-899c-b3b37bca0a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:29.598414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e0a82023-661a-4c16-899c-b3b37bca0a98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:29.706133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a82023-661a-4c16-899c-b3b37bca0a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:08:29.707625 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a82023-661a-4c16-899c-b3b37bca0a98 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:08:29.726085 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 734/1487] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:08:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:08:31.908700 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32c8eb-892d-4c16-80f8-4205e744c206 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:31.909305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32c8eb-892d-4c16-80f8-4205e744c206 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:31.909305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b32c8eb-892d-4c16-80f8-4205e744c206 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:31.910689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 754/1488] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:08:31.915887 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832faf81-cd21-44ed-a012-c2c9d1ccbadb None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:31.916303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832faf81-cd21-44ed-a012-c2c9d1ccbadb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:31.916462 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832faf81-cd21-44ed-a012-c2c9d1ccbadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:31.980444 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-832faf81-cd21-44ed-a012-c2c9d1ccbadb None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:08:31.981680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-832faf81-cd21-44ed-a012-c2c9d1ccbadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:32.034476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:32.080083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 735/1489] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:08:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:08:32.120310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32c8eb-892d-4c16-80f8-4205e744c206 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:08:32.121921 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9b32c8eb-892d-4c16-80f8-4205e744c206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:32.157107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:32.225850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b32c8eb-892d-4c16-80f8-4205e744c206 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:08:32.300007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db0630-04e5-422d-b214-1f1968bd3c77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:08:32.300655 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db0630-04e5-422d-b214-1f1968bd3c77 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:32.300954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db0630-04e5-422d-b214-1f1968bd3c77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:32.301234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db0630-04e5-422d-b214-1f1968bd3c77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:32.302874 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-93db0630-04e5-422d-b214-1f1968bd3c77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:32.381402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:32.411326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db0630-04e5-422d-b214-1f1968bd3c77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:08:32.412767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db0630-04e5-422d-b214-1f1968bd3c77 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:08:32.431282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 755/1490] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:08:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:08:34.632898 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdaac8-1b4f-45a1-9ee9-4f2594b4cc5c None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:34.633476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdaac8-1b4f-45a1-9ee9-4f2594b4cc5c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:34.633575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fdaac8-1b4f-45a1-9ee9-4f2594b4cc5c None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:34.634937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 736/1491] 10.222.0.61 () {54 vars in 926 bytes} [Thu Jul 14 10:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:08:34.640181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90089e17-bb39-4db1-914b-fce91cc37fae None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:34.640624 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90089e17-bb39-4db1-914b-fce91cc37fae None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:34.640729 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90089e17-bb39-4db1-914b-fce91cc37fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:34.704028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-90089e17-bb39-4db1-914b-fce91cc37fae None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:08:34.705251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-90089e17-bb39-4db1-914b-fce91cc37fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:34.800948 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 756/1492] 10.222.0.61 () {58 vars in 1023 bytes} [Thu Jul 14 10:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:08:34.840669 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fdaac8-1b4f-45a1-9ee9-4f2594b4cc5c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:08:34.842246 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-01fdaac8-1b4f-45a1-9ee9-4f2594b4cc5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:34.944194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fdaac8-1b4f-45a1-9ee9-4f2594b4cc5c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:08:35.018512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a2837c-1d8c-4ec0-9b32-6e1444533837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:08:35.019156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2837c-1d8c-4ec0-9b32-6e1444533837 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:08:35.019436 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2837c-1d8c-4ec0-9b32-6e1444533837 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:08:35.019707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a2837c-1d8c-4ec0-9b32-6e1444533837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:08:35.021379 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d2a2837c-1d8c-4ec0-9b32-6e1444533837 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:08:35.032301 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:35.131492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a2837c-1d8c-4ec0-9b32-6e1444533837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:08:35.132915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a2837c-1d8c-4ec0-9b32-6e1444533837 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:08:35.151484 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 737/1493] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:08:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:08:44.799190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=82 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:08:45.148734 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=50 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:10:00.564705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a3a99-008a-4c68-ae8e-3584d77d990c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:00.565308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a3a99-008a-4c68-ae8e-3584d77d990c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:00.565308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a3a99-008a-4c68-ae8e-3584d77d990c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:00.641891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d24a3a99-008a-4c68-ae8e-3584d77d990c None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:10:00.643096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d24a3a99-008a-4c68-ae8e-3584d77d990c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:00.737979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 757/1494] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:10:00.746517 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc96ee-360d-4e8f-adc0-e7900e285ad7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:00.747032 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc96ee-360d-4e8f-adc0-e7900e285ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:00.747139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cc96ee-360d-4e8f-adc0-e7900e285ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:00.820723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a5cc96ee-360d-4e8f-adc0-e7900e285ad7 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:10:00.821894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a5cc96ee-360d-4e8f-adc0-e7900e285ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:00.918533 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 738/1495] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:10:00.923323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a3a99-008a-4c68-ae8e-3584d77d990c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:10:00.924539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d24a3a99-008a-4c68-ae8e-3584d77d990c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:00.935895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a3a99-008a-4c68-ae8e-3584d77d990c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:10:00.996627 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:10:01.030869 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6b6d5-1f03-4602-adb9-14b17d24418f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:10:01.031613 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6b6d5-1f03-4602-adb9-14b17d24418f admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:01.032076 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6b6d5-1f03-4602-adb9-14b17d24418f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:01.032383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace6b6d5-1f03-4602-adb9-14b17d24418f admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:01.033505 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6b6d5-1f03-4602-adb9-14b17d24418f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:10:01.033877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6b6d5-1f03-4602-adb9-14b17d24418f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:10:01.035921 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-ace6b6d5-1f03-4602-adb9-14b17d24418f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:10:01.035921 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Jul 14 10:10:01.035921 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: Jul 14 10:10:01.037202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace6b6d5-1f03-4602-adb9-14b17d24418f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:10:01.048348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 758/1496] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:10:01.056592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc96ee-360d-4e8f-adc0-e7900e285ad7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:10:01.065688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cc96ee-360d-4e8f-adc0-e7900e285ad7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:10:01.127155 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:10:01.163418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e107388-2de1-44e8-8b70-dbf3b6f36134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:10:01.164084 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e107388-2de1-44e8-8b70-dbf3b6f36134 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:01.164370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e107388-2de1-44e8-8b70-dbf3b6f36134 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:01.164642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e107388-2de1-44e8-8b70-dbf3b6f36134 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:01.165370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e107388-2de1-44e8-8b70-dbf3b6f36134 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:10:01.167096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: WARNING py.warnings [None req-7e107388-2de1-44e8-8b70-dbf3b6f36134 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:10:01.167096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: warnings.warn(msg) Jul 14 10:10:01.167096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: Jul 14 10:10:01.168275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e107388-2de1-44e8-8b70-dbf3b6f36134 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:10:01.223121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 739/1497] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:10:01 2022] POST /identity/v3/projects => generated 375 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:10:01.227317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6b6d5-1f03-4602-adb9-14b17d24418f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:10:01.228879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ace6b6d5-1f03-4602-adb9-14b17d24418f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:01.263461 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace6b6d5-1f03-4602-adb9-14b17d24418f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:10:01.358059 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13313ba2-dee9-4ca1-acdb-312847ab5d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:10:01.358746 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13313ba2-dee9-4ca1-acdb-312847ab5d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:01.359033 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13313ba2-dee9-4ca1-acdb-312847ab5d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:01.360279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13313ba2-dee9-4ca1-acdb-312847ab5d2d admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:01.361388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13313ba2-dee9-4ca1-acdb-312847ab5d2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:10:01.362402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-13313ba2-dee9-4ca1-acdb-312847ab5d2d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:10:01.362402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Jul 14 10:10:01.362402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: Jul 14 10:10:01.363649 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13313ba2-dee9-4ca1-acdb-312847ab5d2d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:10:01.404632 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 759/1498] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:10:01 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:10:01.408693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e107388-2de1-44e8-8b70-dbf3b6f36134 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:10:01.410221 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7e107388-2de1-44e8-8b70-dbf3b6f36134 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:01.471553 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e107388-2de1-44e8-8b70-dbf3b6f36134 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:10:01.554205 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:10:01.565664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d897f0-780a-49a5-b4d9-128ea5a2802b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:10:01.566926 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d897f0-780a-49a5-b4d9-128ea5a2802b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:01.567202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d897f0-780a-49a5-b4d9-128ea5a2802b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:01.567475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d897f0-780a-49a5-b4d9-128ea5a2802b admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:01.568283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d897f0-780a-49a5-b4d9-128ea5a2802b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:10:01.569685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: WARNING py.warnings [None req-13d897f0-780a-49a5-b4d9-128ea5a2802b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:10:01.569685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: warnings.warn(msg) Jul 14 10:10:01.569685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: Jul 14 10:10:01.571022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d897f0-780a-49a5-b4d9-128ea5a2802b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:10:01.584662 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 740/1499] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:10:01.588753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13313ba2-dee9-4ca1-acdb-312847ab5d2d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:10:01.590319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-13313ba2-dee9-4ca1-acdb-312847ab5d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:01.601856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13313ba2-dee9-4ca1-acdb-312847ab5d2d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:10:01.626682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:10:01.696597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159deeb4-2d88-4dae-9138-59a4bc46eb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:10:01.697226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159deeb4-2d88-4dae-9138-59a4bc46eb60 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:01.697600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159deeb4-2d88-4dae-9138-59a4bc46eb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:01.697864 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159deeb4-2d88-4dae-9138-59a4bc46eb60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:01.698596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159deeb4-2d88-4dae-9138-59a4bc46eb60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:10:01.699603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-159deeb4-2d88-4dae-9138-59a4bc46eb60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:10:01.699603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Jul 14 10:10:01.699603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: Jul 14 10:10:01.700472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159deeb4-2d88-4dae-9138-59a4bc46eb60 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:10:01.712845 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 760/1500] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:10:01.717022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d897f0-780a-49a5-b4d9-128ea5a2802b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:10:01.718555 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-13d897f0-780a-49a5-b4d9-128ea5a2802b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:01.751301 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d897f0-780a-49a5-b4d9-128ea5a2802b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:10:01.846318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73661478-244c-495e-8ce8-dbabbc67638a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:10:01.847230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73661478-244c-495e-8ce8-dbabbc67638a admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:01.847513 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73661478-244c-495e-8ce8-dbabbc67638a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:01.847784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73661478-244c-495e-8ce8-dbabbc67638a admin admin] PATH_INFO: `/v3/projects/d44e33ab204741229df29981f500cd8c/users/0a5166b2847f49fbb961a85c73ac484b/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:01.870036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73661478-244c-495e-8ce8-dbabbc67638a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d44e33ab204741229df29981f500cd8c, user_id=0a5166b2847f49fbb961a85c73ac484b, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:10:01.871060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: WARNING py.warnings [None req-73661478-244c-495e-8ce8-dbabbc67638a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:10:01.871060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: warnings.warn(msg) Jul 14 10:10:01.871060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: Jul 14 10:10:01.873017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73661478-244c-495e-8ce8-dbabbc67638a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:10:01.896992 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 741/1501] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:10:01 2022] PUT /identity/v3/projects/d44e33ab204741229df29981f500cd8c/users/0a5166b2847f49fbb961a85c73ac484b/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:10:01.909276 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5b219-584b-436f-8e77-c1ef625143ca None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:01.909561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5b219-584b-436f-8e77-c1ef625143ca None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:01.910747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d5b219-584b-436f-8e77-c1ef625143ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:01.965170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-42d5b219-584b-436f-8e77-c1ef625143ca None None] MFA Rules not processed for user `0a5166b2847f49fbb961a85c73ac484b`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:10:01.966298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-42d5b219-584b-436f-8e77-c1ef625143ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:02.043177 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 761/1502] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:10:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:10:02.056939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2b9ef-c9f1-4869-b5ad-b7c102044957 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:02.057241 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2b9ef-c9f1-4869-b5ad-b7c102044957 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:02.057520 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a2b9ef-c9f1-4869-b5ad-b7c102044957 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:02.058973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 742/1503] 10.222.0.61 () {54 vars in 963 bytes} [Thu Jul 14 10:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:10:02.063955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276178d4-ba91-4d99-986b-71164c2d1d34 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:02.064247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276178d4-ba91-4d99-986b-71164c2d1d34 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:02.064553 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276178d4-ba91-4d99-986b-71164c2d1d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:02.093885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:10:02.138118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-276178d4-ba91-4d99-986b-71164c2d1d34 None None] MFA Rules not processed for user `52e3fbd0400a49a69932ce426218f4a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:10:02.139325 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-276178d4-ba91-4d99-986b-71164c2d1d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:02.213641 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 762/1504] 10.222.0.61 () {58 vars in 1060 bytes} [Thu Jul 14 10:10:02 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:10:02.221360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2b9ef-c9f1-4869-b5ad-b7c102044957 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:10:02.223334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f4a2b9ef-c9f1-4869-b5ad-b7c102044957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:02.275791 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:10:02.329737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a2b9ef-c9f1-4869-b5ad-b7c102044957 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:10:02.401625 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b7076-23ed-4900-8331-59e02a60de3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:10:02.402294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b7076-23ed-4900-8331-59e02a60de3a service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:02.402569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b7076-23ed-4900-8331-59e02a60de3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:02.402845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5b7076-23ed-4900-8331-59e02a60de3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:02.404717 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fc5b7076-23ed-4900-8331-59e02a60de3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:02.501979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b7076-23ed-4900-8331-59e02a60de3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:10:02.503339 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5b7076-23ed-4900-8331-59e02a60de3a service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:10:02.506924 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:10:02.522277 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 743/1505] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:10:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:10:02.564226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667e79c-386c-40f0-b099-88171b86f04e None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:02.564511 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667e79c-386c-40f0-b099-88171b86f04e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:02.564786 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667e79c-386c-40f0-b099-88171b86f04e None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:02.566111 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 763/1506] 10.222.0.61 () {52 vars in 829 bytes} [Thu Jul 14 10:10:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:10:02.569771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b7076-23ed-4900-8331-59e02a60de3a service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:10:02.570976 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fc5b7076-23ed-4900-8331-59e02a60de3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:02.675860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5b7076-23ed-4900-8331-59e02a60de3a service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:10:02.768187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf41b60-6aee-4e9b-b510-6f43c9103e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:10:02.769053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf41b60-6aee-4e9b-b510-6f43c9103e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:02.769831 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf41b60-6aee-4e9b-b510-6f43c9103e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:02.770102 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf41b60-6aee-4e9b-b510-6f43c9103e4e admin admin] PATH_INFO: `/v3/users/0a5166b2847f49fbb961a85c73ac484b` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:02.782314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf41b60-6aee-4e9b-b510-6f43c9103e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a5166b2847f49fbb961a85c73ac484b)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:10:02.783339 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: WARNING py.warnings [None req-bdf41b60-6aee-4e9b-b510-6f43c9103e4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:10:02.783339 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: warnings.warn(msg) Jul 14 10:10:02.783339 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: Jul 14 10:10:02.784541 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf41b60-6aee-4e9b-b510-6f43c9103e4e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:10:02.794160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:10:02.846503 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-bdf41b60-6aee-4e9b-b510-6f43c9103e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a5166b2847f49fbb961a85c73ac484b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:10:02.846966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-bdf41b60-6aee-4e9b-b510-6f43c9103e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a5166b2847f49fbb961a85c73ac484b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:10:02.866160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-bdf41b60-6aee-4e9b-b510-6f43c9103e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a5166b2847f49fbb961a85c73ac484b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:10:02.878125 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-bdf41b60-6aee-4e9b-b510-6f43c9103e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a5166b2847f49fbb961a85c73ac484b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:10:02.888588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-bdf41b60-6aee-4e9b-b510-6f43c9103e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a5166b2847f49fbb961a85c73ac484b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:10:02.895338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 744/1507] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:10:02 2022] DELETE /identity/v3/users/0a5166b2847f49fbb961a85c73ac484b => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:10:02.899754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49216c34-8a5a-49e4-8d40-8c7d6a36e427 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:02.900255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49216c34-8a5a-49e4-8d40-8c7d6a36e427 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:02.900365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49216c34-8a5a-49e4-8d40-8c7d6a36e427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:02.954615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-49216c34-8a5a-49e4-8d40-8c7d6a36e427 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:10:02.955819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-49216c34-8a5a-49e4-8d40-8c7d6a36e427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:03.029671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:10:03.042805 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 764/1508] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:10:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:10:03.052346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf41b60-6aee-4e9b-b510-6f43c9103e4e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:10:03.053612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-bdf41b60-6aee-4e9b-b510-6f43c9103e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:03.151278 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf41b60-6aee-4e9b-b510-6f43c9103e4e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:10:03.205829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:10:03.225719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02da87b-b8ad-4675-b92e-78a86313ea16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:10:03.226366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02da87b-b8ad-4675-b92e-78a86313ea16 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:03.226648 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02da87b-b8ad-4675-b92e-78a86313ea16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:03.226923 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02da87b-b8ad-4675-b92e-78a86313ea16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:03.228574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f02da87b-b8ad-4675-b92e-78a86313ea16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:03.337010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02da87b-b8ad-4675-b92e-78a86313ea16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:10:03.338397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02da87b-b8ad-4675-b92e-78a86313ea16 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:10:03.357186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 745/1509] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:10:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:10:03.682670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49216c34-8a5a-49e4-8d40-8c7d6a36e427 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:10:03.683902 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-49216c34-8a5a-49e4-8d40-8c7d6a36e427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:10:03.695863 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49216c34-8a5a-49e4-8d40-8c7d6a36e427 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:10:03.796454 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c4db7a-7d64-4c43-91d3-68bc173632ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:10:03.797360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4db7a-7d64-4c43-91d3-68bc173632ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:10:03.797643 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4db7a-7d64-4c43-91d3-68bc173632ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:10:03.797918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c4db7a-7d64-4c43-91d3-68bc173632ed admin admin] PATH_INFO: `/v3/projects/d44e33ab204741229df29981f500cd8c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:10:03.809068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4db7a-7d64-4c43-91d3-68bc173632ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=d44e33ab204741229df29981f500cd8c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:10:03.810097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-b0c4db7a-7d64-4c43-91d3-68bc173632ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:10:03.810097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Jul 14 10:10:03.810097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: Jul 14 10:10:03.811280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c4db7a-7d64-4c43-91d3-68bc173632ed admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:10:03.870715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b0c4db7a-7d64-4c43-91d3-68bc173632ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd44e33ab204741229df29981f500cd8c', 'request_id': 'req-b0c4db7a-7d64-4c43-91d3-68bc173632ed'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:10:03.884138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b0c4db7a-7d64-4c43-91d3-68bc173632ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd44e33ab204741229df29981f500cd8c', 'request_id': 'req-b0c4db7a-7d64-4c43-91d3-68bc173632ed'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:10:03.890548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b0c4db7a-7d64-4c43-91d3-68bc173632ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd44e33ab204741229df29981f500cd8c', 'request_id': 'req-b0c4db7a-7d64-4c43-91d3-68bc173632ed'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:10:03.916182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b0c4db7a-7d64-4c43-91d3-68bc173632ed admin admin] The token cache is being invalidate because project d44e33ab204741229df29981f500cd8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:10:03.916822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b0c4db7a-7d64-4c43-91d3-68bc173632ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:10:03.916959 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b0c4db7a-7d64-4c43-91d3-68bc173632ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:10:03.918371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 765/1510] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:10:03 2022] DELETE /identity/v3/projects/d44e33ab204741229df29981f500cd8c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:10:13.354813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=62 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:10:13.914140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:11:43.076309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e3f32-34ca-4731-8409-ee35e3a90a79 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:43.076906 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e3f32-34ca-4731-8409-ee35e3a90a79 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:43.076906 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e3f32-34ca-4731-8409-ee35e3a90a79 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:43.078273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:11:43.083235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51111bc7-6eee-4bd7-88f6-45435d19015e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:43.083654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51111bc7-6eee-4bd7-88f6-45435d19015e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:43.083801 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51111bc7-6eee-4bd7-88f6-45435d19015e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:43.161969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-51111bc7-6eee-4bd7-88f6-45435d19015e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:11:43.163178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-51111bc7-6eee-4bd7-88f6-45435d19015e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:11:43.264852 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:11:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:11:43.417412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e3f32-34ca-4731-8409-ee35e3a90a79 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:11:43.419463 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-428e3f32-34ca-4731-8409-ee35e3a90a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:11:43.535396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e3f32-34ca-4731-8409-ee35e3a90a79 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:11:43.540685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2af83-58b7-4fa2-b832-28a92ae1ae94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:11:43.541354 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2af83-58b7-4fa2-b832-28a92ae1ae94 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:43.541731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2af83-58b7-4fa2-b832-28a92ae1ae94 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:43.542035 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2af83-58b7-4fa2-b832-28a92ae1ae94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:43.550809 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2af83-58b7-4fa2-b832-28a92ae1ae94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:11:43.552363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2af83-58b7-4fa2-b832-28a92ae1ae94 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:11:43.561506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 747/1513] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:11:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:11:44.411293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4454b-0cdf-4470-89ff-187eb112f5dd None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:44.411853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4454b-0cdf-4470-89ff-187eb112f5dd None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:44.411853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4454b-0cdf-4470-89ff-187eb112f5dd None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:44.413265 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:11:44.418443 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05b4d0-b635-4263-a74c-4380514a56d9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:44.418850 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05b4d0-b635-4263-a74c-4380514a56d9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:44.419047 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05b4d0-b635-4263-a74c-4380514a56d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:44.438840 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ef05b4d0-b635-4263-a74c-4380514a56d9 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:11:44.440727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ef05b4d0-b635-4263-a74c-4380514a56d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:11:44.448754 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:11:44.606041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4454b-0cdf-4470-89ff-187eb112f5dd None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:11:44.620758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4454b-0cdf-4470-89ff-187eb112f5dd None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:11:44.626306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c18a0a-cb5f-403b-88ba-0eec52635c9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:11:44.626935 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c18a0a-cb5f-403b-88ba-0eec52635c9f service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:44.627211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c18a0a-cb5f-403b-88ba-0eec52635c9f service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:44.627479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c18a0a-cb5f-403b-88ba-0eec52635c9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:44.635448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c18a0a-cb5f-403b-88ba-0eec52635c9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:11:44.637043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c18a0a-cb5f-403b-88ba-0eec52635c9f service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:11:44.646191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 768/1516] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:11:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:11:45.082097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e5b29-ae01-418e-8335-880df0eb448f None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:45.082385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e5b29-ae01-418e-8335-880df0eb448f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:45.082663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505e5b29-ae01-418e-8335-880df0eb448f None None] PATH_INFO: `/v3` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:45.084007 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 749/1517] 10.222.0.58 () {54 vars in 925 bytes} [Thu Jul 14 10:11:45 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:11:45.089996 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889622ee-f47c-4885-92e9-cbf983248f73 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:45.090319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889622ee-f47c-4885-92e9-cbf983248f73 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:45.090601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889622ee-f47c-4885-92e9-cbf983248f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:45.149783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-889622ee-f47c-4885-92e9-cbf983248f73 None None] MFA Rules not processed for user `ff4c42a4f4f549378e5bc1cd92f801c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:11:45.150960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-889622ee-f47c-4885-92e9-cbf983248f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:11:45.167814 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:11:45.222777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 769/1518] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Jul 14 10:11:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:11:48.958961 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3f4e5-9b93-4e3d-9e97-449d220c80b3 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:48.959477 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3f4e5-9b93-4e3d-9e97-449d220c80b3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:48.959477 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb3f4e5-9b93-4e3d-9e97-449d220c80b3 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:48.960822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 750/1519] 10.222.0.61 () {54 vars in 960 bytes} [Thu Jul 14 10:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:11:48.965976 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69795fa4-2d32-4ebc-9a3c-1d781390f199 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:48.966405 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69795fa4-2d32-4ebc-9a3c-1d781390f199 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:48.966526 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69795fa4-2d32-4ebc-9a3c-1d781390f199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:48.984193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-69795fa4-2d32-4ebc-9a3c-1d781390f199 None None] MFA Rules not processed for user `ff4c42a4f4f549378e5bc1cd92f801c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:11:48.985421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-69795fa4-2d32-4ebc-9a3c-1d781390f199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:11:48.994645 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 770/1520] 10.222.0.61 () {58 vars in 1057 bytes} [Thu Jul 14 10:11:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:11:49.002208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3f4e5-9b93-4e3d-9e97-449d220c80b3 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:11:49.015850 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb3f4e5-9b93-4e3d-9e97-449d220c80b3 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:11:49.020796 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1e842e-5fc8-4812-88f9-9eabcc4fd9a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff4c42a4f4f549378e5bc1cd92f801c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:11:49.021448 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e842e-5fc8-4812-88f9-9eabcc4fd9a9 service placement] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:49.021730 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e842e-5fc8-4812-88f9-9eabcc4fd9a9 service placement] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:49.022001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e842e-5fc8-4812-88f9-9eabcc4fd9a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:49.030386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e842e-5fc8-4812-88f9-9eabcc4fd9a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:11:49.031945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1e842e-5fc8-4812-88f9-9eabcc4fd9a9 service placement] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:11:49.041592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 751/1521] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:11:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:11:55.849660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b66c17-c0c0-47e8-88e7-d290bc63296b None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:55.850263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b66c17-c0c0-47e8-88e7-d290bc63296b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:55.850263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b66c17-c0c0-47e8-88e7-d290bc63296b None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:55.851601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:11:55.856418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ce00f-e4fd-4bc4-a292-7fc00d1eb3d5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:55.856848 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ce00f-e4fd-4bc4-a292-7fc00d1eb3d5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:55.857004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767ce00f-e4fd-4bc4-a292-7fc00d1eb3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:55.875219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-767ce00f-e4fd-4bc4-a292-7fc00d1eb3d5 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:11:55.876413 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-767ce00f-e4fd-4bc4-a292-7fc00d1eb3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:11:55.884543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:11:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:11:56.039397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b66c17-c0c0-47e8-88e7-d290bc63296b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:11:56.055109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b66c17-c0c0-47e8-88e7-d290bc63296b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:11:56.060463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f314cf6f-bc62-40d3-98b1-837ba8e325e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:11:56.061133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314cf6f-bc62-40d3-98b1-837ba8e325e8 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:56.061417 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314cf6f-bc62-40d3-98b1-837ba8e325e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:56.061753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f314cf6f-bc62-40d3-98b1-837ba8e325e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:56.069755 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314cf6f-bc62-40d3-98b1-837ba8e325e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:11:56.071320 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f314cf6f-bc62-40d3-98b1-837ba8e325e8 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:11:56.080519 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 772/1524] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:11:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:11:56.897204 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be7aa4-bfe2-412c-827f-6a344fcf8f71 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:56.897782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be7aa4-bfe2-412c-827f-6a344fcf8f71 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:56.897782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be7aa4-bfe2-412c-827f-6a344fcf8f71 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:56.899125 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 14 10:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:11:56.904327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b33915-664a-4e58-916c-8822cd01dd2b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:56.904725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b33915-664a-4e58-916c-8822cd01dd2b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:56.904911 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b33915-664a-4e58-916c-8822cd01dd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:56.922637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e8b33915-664a-4e58-916c-8822cd01dd2b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:11:56.923854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e8b33915-664a-4e58-916c-8822cd01dd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:11:56.932595 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 14 10:11:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:11:57.083937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be7aa4-bfe2-412c-827f-6a344fcf8f71 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:11:57.098815 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be7aa4-bfe2-412c-827f-6a344fcf8f71 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:11:57.104245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd87fc1c-9e36-425b-86eb-695945d24f56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:11:57.104879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87fc1c-9e36-425b-86eb-695945d24f56 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:11:57.105190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87fc1c-9e36-425b-86eb-695945d24f56 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:11:57.105464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd87fc1c-9e36-425b-86eb-695945d24f56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:11:57.114324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87fc1c-9e36-425b-86eb-695945d24f56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:11:57.115896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd87fc1c-9e36-425b-86eb-695945d24f56 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:11:57.125246 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 754/1527] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:11:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:12:01.211183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d48b07-167e-4e73-9929-661a4975534b None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:12:01.211723 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d48b07-167e-4e73-9929-661a4975534b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:12:01.211723 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d48b07-167e-4e73-9929-661a4975534b None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:12:01.213138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:12:01.218262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6a815-b522-466e-ab6b-fb593a8fce12 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:12:01.218691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6a815-b522-466e-ab6b-fb593a8fce12 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:12:01.218823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b6a815-b522-466e-ab6b-fb593a8fce12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:12:01.236641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-51b6a815-b522-466e-ab6b-fb593a8fce12 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:12:01.237876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-51b6a815-b522-466e-ab6b-fb593a8fce12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:12:01.245781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:12:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:12:01.318301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d48b07-167e-4e73-9929-661a4975534b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:12:01.319968 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-67d48b07-167e-4e73-9929-661a4975534b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:12:01.398377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d48b07-167e-4e73-9929-661a4975534b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:12:01.402301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75be3a74-fdf6-42d2-962b-372afc914c91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:12:01.402950 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be3a74-fdf6-42d2-962b-372afc914c91 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:12:01.403228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be3a74-fdf6-42d2-962b-372afc914c91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:12:01.403501 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75be3a74-fdf6-42d2-962b-372afc914c91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:12:01.412371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be3a74-fdf6-42d2-962b-372afc914c91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:12:01.413843 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75be3a74-fdf6-42d2-962b-372afc914c91 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:12:01.422994 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 775/1530] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:12:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:12:03.417879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0b92e-7e0a-4b62-b53b-df8599ac7f23 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:12:03.418403 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0b92e-7e0a-4b62-b53b-df8599ac7f23 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:12:03.418521 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0b92e-7e0a-4b62-b53b-df8599ac7f23 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:12:03.419807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:12:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:12:03.424759 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca5429-1155-42f3-9a06-b2388a20fb17 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:12:03.425194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca5429-1155-42f3-9a06-b2388a20fb17 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:12:03.425350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ca5429-1155-42f3-9a06-b2388a20fb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:12:03.443362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-b6ca5429-1155-42f3-9a06-b2388a20fb17 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:12:03.444541 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b6ca5429-1155-42f3-9a06-b2388a20fb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:12:03.452176 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:12:03.487433 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0b92e-7e0a-4b62-b53b-df8599ac7f23 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:12:03.497969 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0b92e-7e0a-4b62-b53b-df8599ac7f23 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:12:03.501389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfb5b2-cd51-4a79-bad5-b364785c5b47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:12:03.502024 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfb5b2-cd51-4a79-bad5-b364785c5b47 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:12:03.502304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfb5b2-cd51-4a79-bad5-b364785c5b47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:12:03.502576 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfb5b2-cd51-4a79-bad5-b364785c5b47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:12:03.511928 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfb5b2-cd51-4a79-bad5-b364785c5b47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:12:03.513385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecfb5b2-cd51-4a79-bad5-b364785c5b47 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:12:03.523514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 757/1533] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:12:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:12:07.136212 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5d79b-219a-4172-a1e1-789ae3676900 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:12:07.136786 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5d79b-219a-4172-a1e1-789ae3676900 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:12:07.136786 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5d79b-219a-4172-a1e1-789ae3676900 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:12:07.138179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:12:07.143390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208926b-82ea-40b7-8bb2-a1f76ca7cf47 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:12:07.143797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208926b-82ea-40b7-8bb2-a1f76ca7cf47 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:12:07.143919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2208926b-82ea-40b7-8bb2-a1f76ca7cf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:12:07.162170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2208926b-82ea-40b7-8bb2-a1f76ca7cf47 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:12:07.163355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2208926b-82ea-40b7-8bb2-a1f76ca7cf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:12:07.171162 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:12:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:12:07.475396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5d79b-219a-4172-a1e1-789ae3676900 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:12:07.477197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b9a5d79b-219a-4172-a1e1-789ae3676900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:12:07.542963 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:12:07.557091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5d79b-219a-4172-a1e1-789ae3676900 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:12:07.560618 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeb8f70-51c0-4e8d-9f63-5863dda8df13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:12:07.561299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb8f70-51c0-4e8d-9f63-5863dda8df13 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:12:07.561583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb8f70-51c0-4e8d-9f63-5863dda8df13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:12:07.561858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeb8f70-51c0-4e8d-9f63-5863dda8df13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:12:07.571318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb8f70-51c0-4e8d-9f63-5863dda8df13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:12:07.572785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeb8f70-51c0-4e8d-9f63-5863dda8df13 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:12:07.582350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 778/1536] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:12:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:12:09.631318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b8978-5815-450f-b3ef-27c6a1057d6e None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:12:09.631845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b8978-5815-450f-b3ef-27c6a1057d6e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:12:09.631845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b8978-5815-450f-b3ef-27c6a1057d6e None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:12:09.633242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 14 10:12:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:12:09.638313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f9f98-7721-4fce-9c62-58d9ed258535 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:12:09.638711 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f9f98-7721-4fce-9c62-58d9ed258535 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:12:09.638878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f9f98-7721-4fce-9c62-58d9ed258535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:12:09.657383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-379f9f98-7721-4fce-9c62-58d9ed258535 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:12:09.658577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-379f9f98-7721-4fce-9c62-58d9ed258535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:12:09.666415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 14 10:12:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:12:09.966557 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b8978-5815-450f-b3ef-27c6a1057d6e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:12:09.968258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fc9b8978-5815-450f-b3ef-27c6a1057d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:12:09.980825 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b8978-5815-450f-b3ef-27c6a1057d6e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:12:09.984267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ed9ee-277e-4b63-b0bd-425e6d88b41b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:12:09.984894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ed9ee-277e-4b63-b0bd-425e6d88b41b service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:12:09.985200 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ed9ee-277e-4b63-b0bd-425e6d88b41b service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:12:09.985468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ed9ee-277e-4b63-b0bd-425e6d88b41b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:12:09.994700 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ed9ee-277e-4b63-b0bd-425e6d88b41b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:12:09.996093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ed9ee-277e-4b63-b0bd-425e6d88b41b service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:12:10.005824 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 760/1539] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:12:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:12:19.652269 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=11 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:12:20.001313 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:40.125098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c8500-c402-48b7-82be-4ca7af7e5c99 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.125758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c8500-c402-48b7-82be-4ca7af7e5c99 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.125758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c8500-c402-48b7-82be-4ca7af7e5c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.150106 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-bf3c8500-c402-48b7-82be-4ca7af7e5c99 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:40.151325 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bf3c8500-c402-48b7-82be-4ca7af7e5c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:40.169916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 780/1540] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:40.178451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd76521-f7c0-4ffa-ac95-f61a7414a6c5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.179673 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd76521-f7c0-4ffa-ac95-f61a7414a6c5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.179673 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd76521-f7c0-4ffa-ac95-f61a7414a6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.205468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-bcd76521-f7c0-4ffa-ac95-f61a7414a6c5 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:40.206677 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-bcd76521-f7c0-4ffa-ac95-f61a7414a6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:40.221265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 761/1541] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:40.226364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c8500-c402-48b7-82be-4ca7af7e5c99 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:40.238465 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c8500-c402-48b7-82be-4ca7af7e5c99 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:40.244042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c78a1ce-dffb-4826-9363-3293a450f907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:40.244042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78a1ce-dffb-4826-9363-3293a450f907 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.244042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78a1ce-dffb-4826-9363-3293a450f907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.244042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c78a1ce-dffb-4826-9363-3293a450f907 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.244421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78a1ce-dffb-4826-9363-3293a450f907 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:13:40.244421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78a1ce-dffb-4826-9363-3293a450f907 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:40.249964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c78a1ce-dffb-4826-9363-3293a450f907 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:40.266614 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 781/1542] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:13:40.277696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd76521-f7c0-4ffa-ac95-f61a7414a6c5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:40.287380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd76521-f7c0-4ffa-ac95-f61a7414a6c5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:40.290921 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080dbfda-75d0-4083-82c2-d8c1d9a6f1bb None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.291215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080dbfda-75d0-4083-82c2-d8c1d9a6f1bb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.291494 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080dbfda-75d0-4083-82c2-d8c1d9a6f1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.298671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3ceb8-cfab-4275-98c4-035d4f42a5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:40.299368 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3ceb8-cfab-4275-98c4-035d4f42a5a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.299651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3ceb8-cfab-4275-98c4-035d4f42a5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.299929 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed3ceb8-cfab-4275-98c4-035d4f42a5a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.300653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3ceb8-cfab-4275-98c4-035d4f42a5a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:40.307431 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed3ceb8-cfab-4275-98c4-035d4f42a5a2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:40.330068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-080dbfda-75d0-4083-82c2-d8c1d9a6f1bb None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:40.331402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-080dbfda-75d0-4083-82c2-d8c1d9a6f1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:40.355567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 782/1543] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:40.368454 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c6991-4196-4e85-8bb1-43a0cbafb6cd None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.368819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c6991-4196-4e85-8bb1-43a0cbafb6cd None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.369037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1c6991-4196-4e85-8bb1-43a0cbafb6cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.400387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 762/1544] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/projects => generated 371 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:40.405987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3ceb8-cfab-4275-98c4-035d4f42a5a2 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:40.421705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2a1c6991-4196-4e85-8bb1-43a0cbafb6cd None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:40.422940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2a1c6991-4196-4e85-8bb1-43a0cbafb6cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:40.544952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 783/1545] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:40.544952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad243d81-3a3c-4869-8354-9d670a78a70c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.544952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad243d81-3a3c-4869-8354-9d670a78a70c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.544952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad243d81-3a3c-4869-8354-9d670a78a70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.548061 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed3ceb8-cfab-4275-98c4-035d4f42a5a2 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:40.553253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9f626e-ac45-4aa6-ace1-0dca42b26d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:40.554224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f626e-ac45-4aa6-ace1-0dca42b26d95 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.554566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f626e-ac45-4aa6-ace1-0dca42b26d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.554912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9f626e-ac45-4aa6-ace1-0dca42b26d95 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.555665 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f626e-ac45-4aa6-ace1-0dca42b26d95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:40.557056 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: WARNING py.warnings [None req-0a9f626e-ac45-4aa6-ace1-0dca42b26d95 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:13:40.557056 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: warnings.warn(msg) Jul 14 10:13:40.557056 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: Jul 14 10:13:40.558653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9f626e-ac45-4aa6-ace1-0dca42b26d95 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:40.565858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ad243d81-3a3c-4869-8354-9d670a78a70c None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:40.567310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ad243d81-3a3c-4869-8354-9d670a78a70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:40.577028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 784/1546] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:40.578667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f953c9-6ffa-4df6-bb0c-9ba32f3976db None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.579159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f953c9-6ffa-4df6-bb0c-9ba32f3976db None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.579602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f953c9-6ffa-4df6-bb0c-9ba32f3976db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.594267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 763/1547] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:40.595664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f1af3-2fea-46ff-bc65-17b544569fd7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.595937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f1af3-2fea-46ff-bc65-17b544569fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.596214 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f1af3-2fea-46ff-bc65-17b544569fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.603080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-39f953c9-6ffa-4df6-bb0c-9ba32f3976db None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:40.605939 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-39f953c9-6ffa-4df6-bb0c-9ba32f3976db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:40.615732 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 785/1548] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:40.621044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-593f1af3-2fea-46ff-bc65-17b544569fd7 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:40.621044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-593f1af3-2fea-46ff-bc65-17b544569fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:40.621513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872f870-b91d-486f-b0d2-1c447f78d645 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.623176 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872f870-b91d-486f-b0d2-1c447f78d645 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.623474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872f870-b91d-486f-b0d2-1c447f78d645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.637622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 764/1549] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:40.638799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f1af3-2fea-46ff-bc65-17b544569fd7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:40.654696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f1af3-2fea-46ff-bc65-17b544569fd7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:40.658076 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7872f870-b91d-486f-b0d2-1c447f78d645 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:40.659680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e31a9-6a8d-41fe-bb20-eced4c41946e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:40.660190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7872f870-b91d-486f-b0d2-1c447f78d645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:40.660390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e31a9-6a8d-41fe-bb20-eced4c41946e admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.660614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e31a9-6a8d-41fe-bb20-eced4c41946e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.660885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051e31a9-6a8d-41fe-bb20-eced4c41946e admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.661723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e31a9-6a8d-41fe-bb20-eced4c41946e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:13:40.662017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e31a9-6a8d-41fe-bb20-eced4c41946e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:40.663972 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: WARNING py.warnings [None req-051e31a9-6a8d-41fe-bb20-eced4c41946e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:13:40.663972 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: warnings.warn(msg) Jul 14 10:13:40.663972 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: Jul 14 10:13:40.664851 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051e31a9-6a8d-41fe-bb20-eced4c41946e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:40.673564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 786/1550] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:40.675479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7285e-36f3-4c5f-a10a-580188f46987 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.676007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7285e-36f3-4c5f-a10a-580188f46987 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.676696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb7285e-36f3-4c5f-a10a-580188f46987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.683375 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 765/1551] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:13:40.684286 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e31a9-6a8d-41fe-bb20-eced4c41946e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:40.695746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051e31a9-6a8d-41fe-bb20-eced4c41946e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:40.700293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-adb7285e-36f3-4c5f-a10a-580188f46987 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:40.700565 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c17d2-5183-4a1d-9e92-b23ed3cfd2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:40.701276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c17d2-5183-4a1d-9e92-b23ed3cfd2ab admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.701559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c17d2-5183-4a1d-9e92-b23ed3cfd2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.701837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c17d2-5183-4a1d-9e92-b23ed3cfd2ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.702572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c17d2-5183-4a1d-9e92-b23ed3cfd2ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:40.703733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-adb7285e-36f3-4c5f-a10a-580188f46987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:40.704169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c17d2-5183-4a1d-9e92-b23ed3cfd2ab admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:40.715904 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 787/1552] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:40.717772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d0dc4-5422-4ea9-bcbc-cfe25456a7d8 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.718285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d0dc4-5422-4ea9-bcbc-cfe25456a7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.718772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183d0dc4-5422-4ea9-bcbc-cfe25456a7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.721404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 766/1553] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:40.722748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3a8d4-25a5-4a79-9b89-d89b0e9945d4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.723038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3a8d4-25a5-4a79-9b89-d89b0e9945d4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.723321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d3a8d4-25a5-4a79-9b89-d89b0e9945d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.740723 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-183d0dc4-5422-4ea9-bcbc-cfe25456a7d8 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:40.742192 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-183d0dc4-5422-4ea9-bcbc-cfe25456a7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:40.745065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c0d3a8d4-25a5-4a79-9b89-d89b0e9945d4 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:40.746362 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c0d3a8d4-25a5-4a79-9b89-d89b0e9945d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:40.750258 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 788/1554] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:40.751458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2f5a9-e251-40b3-8d66-5090ae42fe38 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.751742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2f5a9-e251-40b3-8d66-5090ae42fe38 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.752016 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f2f5a9-e251-40b3-8d66-5090ae42fe38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.754340 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 767/1555] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:40.755664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba99fd4-8a66-4572-8c4c-f9d7adbb189f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.755953 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba99fd4-8a66-4572-8c4c-f9d7adbb189f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.756228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba99fd4-8a66-4572-8c4c-f9d7adbb189f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.769891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-15f2f5a9-e251-40b3-8d66-5090ae42fe38 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:40.771086 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-15f2f5a9-e251-40b3-8d66-5090ae42fe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:40.773803 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0ba99fd4-8a66-4572-8c4c-f9d7adbb189f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:40.774949 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0ba99fd4-8a66-4572-8c4c-f9d7adbb189f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:40.779818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 789/1556] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:40.780567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2f5a9-e251-40b3-8d66-5090ae42fe38 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:40.790226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 768/1557] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:40.790467 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f2f5a9-e251-40b3-8d66-5090ae42fe38 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:40.791063 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba99fd4-8a66-4572-8c4c-f9d7adbb189f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:40.795147 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c36674-2b9a-40ee-8571-64f394afc130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:40.795809 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c36674-2b9a-40ee-8571-64f394afc130 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.796092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c36674-2b9a-40ee-8571-64f394afc130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.796367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c36674-2b9a-40ee-8571-64f394afc130 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.797094 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c36674-2b9a-40ee-8571-64f394afc130 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:40.798112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-f5c36674-2b9a-40ee-8571-64f394afc130 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:13:40.798112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Jul 14 10:13:40.798112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: Jul 14 10:13:40.799304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c36674-2b9a-40ee-8571-64f394afc130 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:40.805327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba99fd4-8a66-4572-8c4c-f9d7adbb189f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:40.810955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a38e73d-4b37-41f2-841f-4d471e11bf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:40.811620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38e73d-4b37-41f2-841f-4d471e11bf34 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.811900 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38e73d-4b37-41f2-841f-4d471e11bf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.812299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a38e73d-4b37-41f2-841f-4d471e11bf34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.813066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38e73d-4b37-41f2-841f-4d471e11bf34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:40.814656 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a38e73d-4b37-41f2-841f-4d471e11bf34 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:40.825790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 769/1558] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:40.827220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbc3ff-3c45-422e-ad02-2af0e350774d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.827508 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbc3ff-3c45-422e-ad02-2af0e350774d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.827754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 790/1559] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:40.827894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbc3ff-3c45-422e-ad02-2af0e350774d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.829067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c36674-2b9a-40ee-8571-64f394afc130 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:40.849254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b3bbc3ff-3c45-422e-ad02-2af0e350774d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:40.850442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b3bbc3ff-3c45-422e-ad02-2af0e350774d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:40.891888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c36674-2b9a-40ee-8571-64f394afc130 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:40.896438 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd881ced-6e1d-450b-ab65-fa211a94fb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:40.897130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd881ced-6e1d-450b-ab65-fa211a94fb85 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.897410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd881ced-6e1d-450b-ab65-fa211a94fb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.898241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd881ced-6e1d-450b-ab65-fa211a94fb85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.899004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd881ced-6e1d-450b-ab65-fa211a94fb85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:13:40.899312 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd881ced-6e1d-450b-ab65-fa211a94fb85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:40.900876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:40.901028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd881ced-6e1d-450b-ab65-fa211a94fb85 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:40.912345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 770/1560] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:40.912599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 791/1561] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:13:40.913271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbc3ff-3c45-422e-ad02-2af0e350774d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:40.913518 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd881ced-6e1d-450b-ab65-fa211a94fb85 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:40.926025 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbc3ff-3c45-422e-ad02-2af0e350774d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:40.930815 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04442e-0c09-4718-bec9-f4ea647b9896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:40.931468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04442e-0c09-4718-bec9-f4ea647b9896 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.931747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04442e-0c09-4718-bec9-f4ea647b9896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.932019 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04442e-0c09-4718-bec9-f4ea647b9896 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.932782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04442e-0c09-4718-bec9-f4ea647b9896 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:13:40.933121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04442e-0c09-4718-bec9-f4ea647b9896 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:40.933518 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd881ced-6e1d-450b-ab65-fa211a94fb85 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:40.934715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04442e-0c09-4718-bec9-f4ea647b9896 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:40.938196 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f46969-a948-4e83-acd2-edce6bb21b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:40.938869 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f46969-a948-4e83-acd2-edce6bb21b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.939155 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f46969-a948-4e83-acd2-edce6bb21b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.939481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f46969-a948-4e83-acd2-edce6bb21b86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.940209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f46969-a948-4e83-acd2-edce6bb21b86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:13:40.940517 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f46969-a948-4e83-acd2-edce6bb21b86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:40.942127 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f46969-a948-4e83-acd2-edce6bb21b86 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:40.943631 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:40.947135 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 771/1562] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:13:40.948369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3868ba2-b629-4280-8ba6-f6866515820d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.948653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3868ba2-b629-4280-8ba6-f6866515820d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.949014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3868ba2-b629-4280-8ba6-f6866515820d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.957844 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 792/1563] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:13:40.958766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f46969-a948-4e83-acd2-edce6bb21b86 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:40.969169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f46969-a948-4e83-acd2-edce6bb21b86 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:40.969339 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d3868ba2-b629-4280-8ba6-f6866515820d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:40.970526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d3868ba2-b629-4280-8ba6-f6866515820d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:40.973873 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff517f9-f1f3-4c4c-b087-82d9664393f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:40.974510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff517f9-f1f3-4c4c-b087-82d9664393f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.974814 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff517f9-f1f3-4c4c-b087-82d9664393f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.975210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff517f9-f1f3-4c4c-b087-82d9664393f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:40.975993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff517f9-f1f3-4c4c-b087-82d9664393f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:13:40.976292 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff517f9-f1f3-4c4c-b087-82d9664393f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:40.977896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff517f9-f1f3-4c4c-b087-82d9664393f0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:40.978539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 772/1564] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:40.979286 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3868ba2-b629-4280-8ba6-f6866515820d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:40.988609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3868ba2-b629-4280-8ba6-f6866515820d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:40.989988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 793/1565] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:13:40.990755 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff517f9-f1f3-4c4c-b087-82d9664393f0 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:40.993512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3701f-5582-4786-a980-b550cb6264e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:40.994394 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3701f-5582-4786-a980-b550cb6264e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:40.994678 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3701f-5582-4786-a980-b550cb6264e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:40.995072 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3701f-5582-4786-a980-b550cb6264e2 admin admin] PATH_INFO: `/v3/projects/c3aace1f1bdb4d809577f16539279436/users/23dad834c85e4246b00c3b232b5193e2/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.002200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff517f9-f1f3-4c4c-b087-82d9664393f0 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.007387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b304b-2495-429f-b960-f1e614df9356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.008032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b304b-2495-429f-b960-f1e614df9356 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.008312 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b304b-2495-429f-b960-f1e614df9356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.008585 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394b304b-2495-429f-b960-f1e614df9356 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.009307 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b304b-2495-429f-b960-f1e614df9356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.010313 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3701f-5582-4786-a980-b550cb6264e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3aace1f1bdb4d809577f16539279436, user_id=23dad834c85e4246b00c3b232b5193e2, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.011198 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394b304b-2495-429f-b960-f1e614df9356 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.012903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3701f-5582-4786-a980-b550cb6264e2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.029930 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 773/1566] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:40 2022] PUT /identity/v3/projects/c3aace1f1bdb4d809577f16539279436/users/23dad834c85e4246b00c3b232b5193e2/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:41.031422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b74c26-64e5-449f-bd8e-57ec5201d4e1 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.031713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b74c26-64e5-449f-bd8e-57ec5201d4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.031991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b74c26-64e5-449f-bd8e-57ec5201d4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.039894 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 794/1567] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:13:40 2022] POST /identity/v3/users => generated 289 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:41.041293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b304b-2495-429f-b960-f1e614df9356 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.052468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394b304b-2495-429f-b960-f1e614df9356 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.055044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-31b74c26-64e5-449f-bd8e-57ec5201d4e1 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:41.057258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-31b74c26-64e5-449f-bd8e-57ec5201d4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:41.110108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f82f59-8f95-4238-8a93-d4377b4be9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.110772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f82f59-8f95-4238-8a93-d4377b4be9ae admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.111051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f82f59-8f95-4238-8a93-d4377b4be9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.111326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f82f59-8f95-4238-8a93-d4377b4be9ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.112016 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f82f59-8f95-4238-8a93-d4377b4be9ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.113944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f82f59-8f95-4238-8a93-d4377b4be9ae admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.116987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 774/1568] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:41.117797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b74c26-64e5-449f-bd8e-57ec5201d4e1 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.126639 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b74c26-64e5-449f-bd8e-57ec5201d4e1 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.131242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17c8d5-ce54-4218-aa57-d5a6f368e05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.132399 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17c8d5-ce54-4218-aa57-d5a6f368e05b admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.132675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17c8d5-ce54-4218-aa57-d5a6f368e05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.132970 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17c8d5-ce54-4218-aa57-d5a6f368e05b admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.133654 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17c8d5-ce54-4218-aa57-d5a6f368e05b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.135534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17c8d5-ce54-4218-aa57-d5a6f368e05b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.141530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 795/1569] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:41.143027 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f82f59-8f95-4238-8a93-d4377b4be9ae admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.163226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 775/1570] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:41.164181 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17c8d5-ce54-4218-aa57-d5a6f368e05b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.209063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f82f59-8f95-4238-8a93-d4377b4be9ae admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.224640 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17c8d5-ce54-4218-aa57-d5a6f368e05b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.229508 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cd72f-cb78-4de0-b106-442af944eb82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.230175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cd72f-cb78-4de0-b106-442af944eb82 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.230450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cd72f-cb78-4de0-b106-442af944eb82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.230727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4cd72f-cb78-4de0-b106-442af944eb82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.231460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cd72f-cb78-4de0-b106-442af944eb82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:13:41.231769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cd72f-cb78-4de0-b106-442af944eb82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.233895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4cd72f-cb78-4de0-b106-442af944eb82 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.253656 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 776/1571] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:13:41.253656 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cd72f-cb78-4de0-b106-442af944eb82 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.258053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4cd72f-cb78-4de0-b106-442af944eb82 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.262770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851a2cb-f586-4ea9-999c-6bf61b0562b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.263417 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851a2cb-f586-4ea9-999c-6bf61b0562b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.263698 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851a2cb-f586-4ea9-999c-6bf61b0562b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.263982 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851a2cb-f586-4ea9-999c-6bf61b0562b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.264669 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851a2cb-f586-4ea9-999c-6bf61b0562b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.266607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851a2cb-f586-4ea9-999c-6bf61b0562b0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.272190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:41.280362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f1c3b5-707a-4d6a-8e52-aa3d5ca63601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.281084 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1c3b5-707a-4d6a-8e52-aa3d5ca63601 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.281368 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1c3b5-707a-4d6a-8e52-aa3d5ca63601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.281990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f1c3b5-707a-4d6a-8e52-aa3d5ca63601 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.282500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1c3b5-707a-4d6a-8e52-aa3d5ca63601 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.286669 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f1c3b5-707a-4d6a-8e52-aa3d5ca63601 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.293092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 777/1572] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:41.293992 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851a2cb-f586-4ea9-999c-6bf61b0562b0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.299337 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:41.313968 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 796/1573] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:41.318481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03455d-d3d8-4f98-aa6f-815cdb515b24 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.318770 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03455d-d3d8-4f98-aa6f-815cdb515b24 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.319050 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03455d-d3d8-4f98-aa6f-815cdb515b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.345419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3e03455d-d3d8-4f98-aa6f-815cdb515b24 None None] MFA Rules not processed for user `23dad834c85e4246b00c3b232b5193e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:41.346619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3e03455d-d3d8-4f98-aa6f-815cdb515b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:41.358459 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851a2cb-f586-4ea9-999c-6bf61b0562b0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.410973 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 797/1574] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:41.411828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03455d-d3d8-4f98-aa6f-815cdb515b24 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.417304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2675f-62ad-4b0e-99cc-2c745a9c5d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.417978 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2675f-62ad-4b0e-99cc-2c745a9c5d8e admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.418258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2675f-62ad-4b0e-99cc-2c745a9c5d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.418529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa2675f-62ad-4b0e-99cc-2c745a9c5d8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.419256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2675f-62ad-4b0e-99cc-2c745a9c5d8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.420844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa2675f-62ad-4b0e-99cc-2c745a9c5d8e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.434080 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 778/1575] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:41.434080 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2675f-62ad-4b0e-99cc-2c745a9c5d8e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.443023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa2675f-62ad-4b0e-99cc-2c745a9c5d8e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.448128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42b1e7-e7d2-4c45-b854-1c58f755a57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.448773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42b1e7-e7d2-4c45-b854-1c58f755a57e admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.449085 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42b1e7-e7d2-4c45-b854-1c58f755a57e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.449358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff42b1e7-e7d2-4c45-b854-1c58f755a57e admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.450038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42b1e7-e7d2-4c45-b854-1c58f755a57e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.451922 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff42b1e7-e7d2-4c45-b854-1c58f755a57e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.472259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 779/1576] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/users => generated 286 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:41.473079 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42b1e7-e7d2-4c45-b854-1c58f755a57e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.476394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03455d-d3d8-4f98-aa6f-815cdb515b24 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.481065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c5356-79b6-4359-b9a5-8e25952b5a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.481726 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c5356-79b6-4359-b9a5-8e25952b5a32 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.482007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c5356-79b6-4359-b9a5-8e25952b5a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.482209 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff42b1e7-e7d2-4c45-b854-1c58f755a57e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.482331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43c5356-79b6-4359-b9a5-8e25952b5a32 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.482996 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c5356-79b6-4359-b9a5-8e25952b5a32 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:13:41.483299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c5356-79b6-4359-b9a5-8e25952b5a32 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.484878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43c5356-79b6-4359-b9a5-8e25952b5a32 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.486877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5cecb-b527-4920-835d-14118c42bbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.487512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5cecb-b527-4920-835d-14118c42bbe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.487788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5cecb-b527-4920-835d-14118c42bbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.488057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5cecb-b527-4920-835d-14118c42bbe0 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.488727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5cecb-b527-4920-835d-14118c42bbe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.490735 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5cecb-b527-4920-835d-14118c42bbe0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.496785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 798/1577] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:13:41.497669 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c5356-79b6-4359-b9a5-8e25952b5a32 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.508359 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43c5356-79b6-4359-b9a5-8e25952b5a32 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.514592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 780/1578] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/users => generated 283 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:41.514592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5cecb-b527-4920-835d-14118c42bbe0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.516173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520074f-61e5-4f85-ba0a-01b0a05d0094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.516850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520074f-61e5-4f85-ba0a-01b0a05d0094 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.517157 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520074f-61e5-4f85-ba0a-01b0a05d0094 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.517433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520074f-61e5-4f85-ba0a-01b0a05d0094 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.518121 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520074f-61e5-4f85-ba0a-01b0a05d0094 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.520019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520074f-61e5-4f85-ba0a-01b0a05d0094 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.523132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5cecb-b527-4920-835d-14118c42bbe0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.528110 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc2a19-dd3f-446c-baab-1764846c1f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.529044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc2a19-dd3f-446c-baab-1764846c1f04 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.529287 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc2a19-dd3f-446c-baab-1764846c1f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.529607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc2a19-dd3f-446c-baab-1764846c1f04 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.530323 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc2a19-dd3f-446c-baab-1764846c1f04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.532219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cc2a19-dd3f-446c-baab-1764846c1f04 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.548401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 799/1579] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/projects => generated 393 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:41.549264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520074f-61e5-4f85-ba0a-01b0a05d0094 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.559645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 781/1580] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/users => generated 292 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:41.559645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc2a19-dd3f-446c-baab-1764846c1f04 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.569258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc2a19-dd3f-446c-baab-1764846c1f04 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.595806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:41.614690 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520074f-61e5-4f85-ba0a-01b0a05d0094 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.619896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:41.620992 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5405812-0dc8-4e24-8892-5967c7efcc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.621687 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5405812-0dc8-4e24-8892-5967c7efcc90 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.622012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5405812-0dc8-4e24-8892-5967c7efcc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.622290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5405812-0dc8-4e24-8892-5967c7efcc90 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.622979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5405812-0dc8-4e24-8892-5967c7efcc90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.624942 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5405812-0dc8-4e24-8892-5967c7efcc90 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.628390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c52e84-d80a-452e-96f4-27e63632b95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.629138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c52e84-d80a-452e-96f4-27e63632b95d admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.629425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c52e84-d80a-452e-96f4-27e63632b95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.629700 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c52e84-d80a-452e-96f4-27e63632b95d admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.630426 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c52e84-d80a-452e-96f4-27e63632b95d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.632552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c52e84-d80a-452e-96f4-27e63632b95d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.644558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 800/1581] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:41.645408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5405812-0dc8-4e24-8892-5967c7efcc90 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.655749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5405812-0dc8-4e24-8892-5967c7efcc90 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.660348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b368c71-35da-4c92-bba6-19b75d454485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.661088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b368c71-35da-4c92-bba6-19b75d454485 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.661320 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b368c71-35da-4c92-bba6-19b75d454485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.661597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b368c71-35da-4c92-bba6-19b75d454485 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.662322 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b368c71-35da-4c92-bba6-19b75d454485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.662843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 782/1582] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:41.664423 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c52e84-d80a-452e-96f4-27e63632b95d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.664544 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b368c71-35da-4c92-bba6-19b75d454485 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.674609 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 801/1583] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:41.675501 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b368c71-35da-4c92-bba6-19b75d454485 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.685400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b368c71-35da-4c92-bba6-19b75d454485 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.730807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c52e84-d80a-452e-96f4-27e63632b95d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.735671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebfa6d-97e9-4a23-b737-e2aeaaa9e2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.736341 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebfa6d-97e9-4a23-b737-e2aeaaa9e2b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.736625 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebfa6d-97e9-4a23-b737-e2aeaaa9e2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.736903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bebfa6d-97e9-4a23-b737-e2aeaaa9e2b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.737662 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebfa6d-97e9-4a23-b737-e2aeaaa9e2b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.739286 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bebfa6d-97e9-4a23-b737-e2aeaaa9e2b4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.742870 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f61de-9472-4607-b611-8bb603e4f8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.743525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f61de-9472-4607-b611-8bb603e4f8bf admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.743807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f61de-9472-4607-b611-8bb603e4f8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.744075 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f61de-9472-4607-b611-8bb603e4f8bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.744768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f61de-9472-4607-b611-8bb603e4f8bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.746671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46f61de-9472-4607-b611-8bb603e4f8bf admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.752617 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 783/1584] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:41.752617 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebfa6d-97e9-4a23-b737-e2aeaaa9e2b4 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.760416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bebfa6d-97e9-4a23-b737-e2aeaaa9e2b4 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.765571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2f7a9-d3f5-4bc8-b586-cf398b2ae361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.766260 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2f7a9-d3f5-4bc8-b586-cf398b2ae361 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.766543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2f7a9-d3f5-4bc8-b586-cf398b2ae361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.766814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b2f7a9-d3f5-4bc8-b586-cf398b2ae361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.767559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2f7a9-d3f5-4bc8-b586-cf398b2ae361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.769178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b2f7a9-d3f5-4bc8-b586-cf398b2ae361 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.773198 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 802/1585] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:41.774014 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f61de-9472-4607-b611-8bb603e4f8bf admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.785954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 784/1586] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:41.785954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2f7a9-d3f5-4bc8-b586-cf398b2ae361 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.795873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b2f7a9-d3f5-4bc8-b586-cf398b2ae361 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.835832 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f61de-9472-4607-b611-8bb603e4f8bf admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.840658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06293a8f-5177-4c93-ac91-6ce503ff5eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.841567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06293a8f-5177-4c93-ac91-6ce503ff5eaa admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.841714 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06293a8f-5177-4c93-ac91-6ce503ff5eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.841974 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06293a8f-5177-4c93-ac91-6ce503ff5eaa admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.842631 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06293a8f-5177-4c93-ac91-6ce503ff5eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.844706 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06293a8f-5177-4c93-ac91-6ce503ff5eaa admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.852156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13436021-73cc-4505-88b8-9869d672c1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.852819 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13436021-73cc-4505-88b8-9869d672c1ba admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.853133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13436021-73cc-4505-88b8-9869d672c1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.853411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13436021-73cc-4505-88b8-9869d672c1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.854259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13436021-73cc-4505-88b8-9869d672c1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.855993 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13436021-73cc-4505-88b8-9869d672c1ba admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.865183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 803/1587] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:41.865876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06293a8f-5177-4c93-ac91-6ce503ff5eaa admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.870271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 785/1588] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:41.870271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13436021-73cc-4505-88b8-9869d672c1ba admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.875635 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06293a8f-5177-4c93-ac91-6ce503ff5eaa admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.878029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13436021-73cc-4505-88b8-9869d672c1ba admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.880296 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad866484-21fb-4c4a-bd1e-5e898e634502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.880949 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad866484-21fb-4c4a-bd1e-5e898e634502 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.881243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad866484-21fb-4c4a-bd1e-5e898e634502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.881511 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad866484-21fb-4c4a-bd1e-5e898e634502 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.882255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad866484-21fb-4c4a-bd1e-5e898e634502 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.882654 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b921f-4800-45da-bd8c-d1fd360005aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.883295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b921f-4800-45da-bd8c-d1fd360005aa admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.883578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b921f-4800-45da-bd8c-d1fd360005aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.883820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad866484-21fb-4c4a-bd1e-5e898e634502 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.883922 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b921f-4800-45da-bd8c-d1fd360005aa admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.884524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b921f-4800-45da-bd8c-d1fd360005aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.886425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b921f-4800-45da-bd8c-d1fd360005aa admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.893789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 804/1589] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:41.894660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad866484-21fb-4c4a-bd1e-5e898e634502 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.906292 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad866484-21fb-4c4a-bd1e-5e898e634502 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.910294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 786/1590] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/users => generated 292 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:41.910294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b921f-4800-45da-bd8c-d1fd360005aa admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.911170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56034926-e695-4b68-a2e0-e0483181f567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.911996 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56034926-e695-4b68-a2e0-e0483181f567 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.912279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56034926-e695-4b68-a2e0-e0483181f567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.912554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56034926-e695-4b68-a2e0-e0483181f567 admin admin] PATH_INFO: `/v3/projects/7a5b4dfcb90d421192be6dbc102ca1bb/users/fed44db7dc014719b5ed0399c715435b/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.919771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b921f-4800-45da-bd8c-d1fd360005aa admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.924683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6535b0f-8f81-4b30-98bb-574f4457ed82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:41.925377 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6535b0f-8f81-4b30-98bb-574f4457ed82 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:41.925666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6535b0f-8f81-4b30-98bb-574f4457ed82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:41.925947 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6535b0f-8f81-4b30-98bb-574f4457ed82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:41.926665 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6535b0f-8f81-4b30-98bb-574f4457ed82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.928264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6535b0f-8f81-4b30-98bb-574f4457ed82 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.930209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56034926-e695-4b68-a2e0-e0483181f567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a5b4dfcb90d421192be6dbc102ca1bb, user_id=fed44db7dc014719b5ed0399c715435b, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:41.931242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-56034926-e695-4b68-a2e0-e0483181f567 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:13:41.931242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Jul 14 10:13:41.931242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: Jul 14 10:13:41.933358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56034926-e695-4b68-a2e0-e0483181f567 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:41.940134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 787/1591] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:41.941028 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6535b0f-8f81-4b30-98bb-574f4457ed82 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.942925 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:41.948329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 805/1592] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:41 2022] PUT /identity/v3/projects/7a5b4dfcb90d421192be6dbc102ca1bb/users/fed44db7dc014719b5ed0399c715435b/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:41.949472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56034926-e695-4b68-a2e0-e0483181f567 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:41.951026 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6535b0f-8f81-4b30-98bb-574f4457ed82 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:41.983060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:42.010439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202b362-7d7b-4aeb-bd8a-70abea382ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.011108 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202b362-7d7b-4aeb-bd8a-70abea382ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.011380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202b362-7d7b-4aeb-bd8a-70abea382ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.011649 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3202b362-7d7b-4aeb-bd8a-70abea382ff8 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.012346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202b362-7d7b-4aeb-bd8a-70abea382ff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.014259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3202b362-7d7b-4aeb-bd8a-70abea382ff8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.015606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56034926-e695-4b68-a2e0-e0483181f567 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.020227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79d975-ae2b-4947-8394-2dc247086e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.020891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79d975-ae2b-4947-8394-2dc247086e18 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.021202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79d975-ae2b-4947-8394-2dc247086e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.021487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a79d975-ae2b-4947-8394-2dc247086e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.022212 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79d975-ae2b-4947-8394-2dc247086e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.024821 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a79d975-ae2b-4947-8394-2dc247086e18 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.034635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 788/1593] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:13:41 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:42.035439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202b362-7d7b-4aeb-bd8a-70abea382ff8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.039024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 806/1594] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:42.039879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79d975-ae2b-4947-8394-2dc247086e18 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.047096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3202b362-7d7b-4aeb-bd8a-70abea382ff8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.049542 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a79d975-ae2b-4947-8394-2dc247086e18 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.052446 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa61b3-ca44-4f72-84b4-74610822fc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.053138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa61b3-ca44-4f72-84b4-74610822fc4f admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.053425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa61b3-ca44-4f72-84b4-74610822fc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.053731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa61b3-ca44-4f72-84b4-74610822fc4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.054465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa61b3-ca44-4f72-84b4-74610822fc4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.055152 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e0791-8fc9-4f97-9fb1-3d46a6c38019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.055802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e0791-8fc9-4f97-9fb1-3d46a6c38019 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.056050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa61b3-ca44-4f72-84b4-74610822fc4f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.056169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e0791-8fc9-4f97-9fb1-3d46a6c38019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.056339 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22e0791-8fc9-4f97-9fb1-3d46a6c38019 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.057092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e0791-8fc9-4f97-9fb1-3d46a6c38019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.058731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22e0791-8fc9-4f97-9fb1-3d46a6c38019 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.066311 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 789/1595] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:42.067111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa61b3-ca44-4f72-84b4-74610822fc4f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.068724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 807/1596] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:42.070228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e0791-8fc9-4f97-9fb1-3d46a6c38019 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.079987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22e0791-8fc9-4f97-9fb1-3d46a6c38019 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.080510 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa61b3-ca44-4f72-84b4-74610822fc4f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.084513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794b1c4-c8b1-4403-b95e-f0e827c894f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.085704 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794b1c4-c8b1-4403-b95e-f0e827c894f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.085833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff54c2d7-e3b3-426b-b21e-44631237847a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.085964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794b1c4-c8b1-4403-b95e-f0e827c894f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.086236 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d794b1c4-c8b1-4403-b95e-f0e827c894f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.086453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54c2d7-e3b3-426b-b21e-44631237847a admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.086730 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54c2d7-e3b3-426b-b21e-44631237847a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.086949 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794b1c4-c8b1-4403-b95e-f0e827c894f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.087044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff54c2d7-e3b3-426b-b21e-44631237847a admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.087698 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54c2d7-e3b3-426b-b21e-44631237847a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.088521 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d794b1c4-c8b1-4403-b95e-f0e827c894f2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.089295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff54c2d7-e3b3-426b-b21e-44631237847a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.098575 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 808/1597] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:42.099416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794b1c4-c8b1-4403-b95e-f0e827c894f2 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.099644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 790/1598] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:42.101468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee69915-701e-4b57-93e1-9e3a13fed1e1 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.101751 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee69915-701e-4b57-93e1-9e3a13fed1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.102030 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee69915-701e-4b57-93e1-9e3a13fed1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.111181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d794b1c4-c8b1-4403-b95e-f0e827c894f2 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.115765 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49074ffd-6a2c-4d5a-86ee-f7d819b119ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.116670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49074ffd-6a2c-4d5a-86ee-f7d819b119ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.116981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49074ffd-6a2c-4d5a-86ee-f7d819b119ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.117283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49074ffd-6a2c-4d5a-86ee-f7d819b119ea admin admin] PATH_INFO: `/v3/projects/a047dd397bbe42c487a4a2dce8cf6517/users/b8e0c072b7a24d30893957523763e851/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.118653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2ee69915-701e-4b57-93e1-9e3a13fed1e1 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:42.119880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2ee69915-701e-4b57-93e1-9e3a13fed1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:42.127418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 791/1599] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:42.128195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee69915-701e-4b57-93e1-9e3a13fed1e1 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.132318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49074ffd-6a2c-4d5a-86ee-f7d819b119ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a047dd397bbe42c487a4a2dce8cf6517, user_id=b8e0c072b7a24d30893957523763e851, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.135202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49074ffd-6a2c-4d5a-86ee-f7d819b119ea admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.144090 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee69915-701e-4b57-93e1-9e3a13fed1e1 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.149169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 809/1600] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:42 2022] PUT /identity/v3/projects/a047dd397bbe42c487a4a2dce8cf6517/users/b8e0c072b7a24d30893957523763e851/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:42.150035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49074ffd-6a2c-4d5a-86ee-f7d819b119ea admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.209905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeb425-fbca-48d9-84a4-8b1c0e0103c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.211119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeb425-fbca-48d9-84a4-8b1c0e0103c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.211395 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeb425-fbca-48d9-84a4-8b1c0e0103c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.211662 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebeb425-fbca-48d9-84a4-8b1c0e0103c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.212052 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49074ffd-6a2c-4d5a-86ee-f7d819b119ea admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.212409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebeb425-fbca-48d9-84a4-8b1c0e0103c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.214023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebeb425-fbca-48d9-84a4-8b1c0e0103c6 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.216693 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f152f-f630-405b-83ad-b437c6c10487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.217604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f152f-f630-405b-83ad-b437c6c10487 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.217886 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f152f-f630-405b-83ad-b437c6c10487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.218160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7f152f-f630-405b-83ad-b437c6c10487 admin admin] PATH_INFO: `/v3/projects/38008d54e5554fb78cc9ced7e6aa2967/users/4d2739e2821d472da275b8bca6a1bd87/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.223609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 792/1601] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:42.224373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeb425-fbca-48d9-84a4-8b1c0e0103c6 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.233109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f152f-f630-405b-83ad-b437c6c10487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38008d54e5554fb78cc9ced7e6aa2967, user_id=4d2739e2821d472da275b8bca6a1bd87, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.235755 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7f152f-f630-405b-83ad-b437c6c10487 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.236910 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebeb425-fbca-48d9-84a4-8b1c0e0103c6 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.242352 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b3878-dcf2-4891-9c21-0025dd9c5ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.243276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b3878-dcf2-4891-9c21-0025dd9c5ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.243522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b3878-dcf2-4891-9c21-0025dd9c5ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.243832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b3878-dcf2-4891-9c21-0025dd9c5ff9 admin admin] PATH_INFO: `/v3/projects/5056f0a32e37430ea3ea63d9cb69bf2b/users/bff5f2d1367545f2b4c77fe0d9f095ab/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.248972 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 810/1602] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:42 2022] PUT /identity/v3/projects/38008d54e5554fb78cc9ced7e6aa2967/users/4d2739e2821d472da275b8bca6a1bd87/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:42.250163 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f152f-f630-405b-83ad-b437c6c10487 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.263427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b3878-dcf2-4891-9c21-0025dd9c5ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5056f0a32e37430ea3ea63d9cb69bf2b, user_id=bff5f2d1367545f2b4c77fe0d9f095ab, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.266120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b3878-dcf2-4891-9c21-0025dd9c5ff9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.279913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 793/1603] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:42 2022] PUT /identity/v3/projects/5056f0a32e37430ea3ea63d9cb69bf2b/users/bff5f2d1367545f2b4c77fe0d9f095ab/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:42.280691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b3878-dcf2-4891-9c21-0025dd9c5ff9 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.293998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:42.323283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7f152f-f630-405b-83ad-b437c6c10487 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.342953 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b3878-dcf2-4891-9c21-0025dd9c5ff9 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.347601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72e534-ddf2-456a-ab1d-07f5b1002f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.348485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72e534-ddf2-456a-ab1d-07f5b1002f95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.348764 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72e534-ddf2-456a-ab1d-07f5b1002f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.349071 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72e534-ddf2-456a-ab1d-07f5b1002f95 admin admin] PATH_INFO: `/v3/projects/1c8ba47db4ef4546944bc305b7e580d5/users/027695398c144b12aac9aeebca663c44/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.360320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:42.365038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c72e534-ddf2-456a-ab1d-07f5b1002f95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c8ba47db4ef4546944bc305b7e580d5, user_id=027695398c144b12aac9aeebca663c44, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.368759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c72e534-ddf2-456a-ab1d-07f5b1002f95 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.380490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede05b1-6266-4091-b707-96878b00292b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.381203 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede05b1-6266-4091-b707-96878b00292b admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.381484 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede05b1-6266-4091-b707-96878b00292b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.381759 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede05b1-6266-4091-b707-96878b00292b admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.382363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 794/1604] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:42 2022] PUT /identity/v3/projects/1c8ba47db4ef4546944bc305b7e580d5/users/027695398c144b12aac9aeebca663c44/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:42.382493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede05b1-6266-4091-b707-96878b00292b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.383130 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72e534-ddf2-456a-ab1d-07f5b1002f95 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.384077 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede05b1-6266-4091-b707-96878b00292b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.399860 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 811/1605] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:42.399860 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede05b1-6266-4091-b707-96878b00292b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.448705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c72e534-ddf2-456a-ab1d-07f5b1002f95 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.453735 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ca5efe-c2df-4020-8098-ed50e05922a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.454215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede05b1-6266-4091-b707-96878b00292b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.454653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca5efe-c2df-4020-8098-ed50e05922a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.454931 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca5efe-c2df-4020-8098-ed50e05922a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.455200 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ca5efe-c2df-4020-8098-ed50e05922a4 admin admin] PATH_INFO: `/v3/projects/5ea23db89af74a5e8577533cf5c00bb9/users/70b000c2b8f741eeb8199a98f939acbc/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.469626 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca5efe-c2df-4020-8098-ed50e05922a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ea23db89af74a5e8577533cf5c00bb9, user_id=70b000c2b8f741eeb8199a98f939acbc, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.472878 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ca5efe-c2df-4020-8098-ed50e05922a4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.486479 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 795/1606] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:42 2022] PUT /identity/v3/projects/5ea23db89af74a5e8577533cf5c00bb9/users/70b000c2b8f741eeb8199a98f939acbc/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:42.487732 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec28691-0b63-4700-aebd-161d36967154 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.488023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec28691-0b63-4700-aebd-161d36967154 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.488299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec28691-0b63-4700-aebd-161d36967154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.514497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ccb12f-5588-4ec7-bcb0-80ee7ac5957a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.515152 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccb12f-5588-4ec7-bcb0-80ee7ac5957a service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.515328 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-aec28691-0b63-4700-aebd-161d36967154 None None] MFA Rules not processed for user `b8e0c072b7a24d30893957523763e851`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:42.515456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccb12f-5588-4ec7-bcb0-80ee7ac5957a service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.515696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccb12f-5588-4ec7-bcb0-80ee7ac5957a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.516547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-aec28691-0b63-4700-aebd-161d36967154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:42.524902 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ccb12f-5588-4ec7-bcb0-80ee7ac5957a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.526478 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ccb12f-5588-4ec7-bcb0-80ee7ac5957a service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.536977 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 812/1607] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:13:42.537910 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ccb12f-5588-4ec7-bcb0-80ee7ac5957a service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.548625 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ccb12f-5588-4ec7-bcb0-80ee7ac5957a service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.588915 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:42.596020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 796/1608] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:42.597367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c509c-4698-432f-b681-a1c3ef3b7e0f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.597655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c509c-4698-432f-b681-a1c3ef3b7e0f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.597936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c509c-4698-432f-b681-a1c3ef3b7e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.630451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-347c509c-4698-432f-b681-a1c3ef3b7e0f None None] MFA Rules not processed for user `4d2739e2821d472da275b8bca6a1bd87`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:42.631876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-347c509c-4698-432f-b681-a1c3ef3b7e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:42.636112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f0af5a-9ca8-45d3-91bc-203a12f90c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.637178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0af5a-9ca8-45d3-91bc-203a12f90c3f admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.638183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0af5a-9ca8-45d3-91bc-203a12f90c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.639015 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f0af5a-9ca8-45d3-91bc-203a12f90c3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.640861 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0af5a-9ca8-45d3-91bc-203a12f90c3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.644400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f0af5a-9ca8-45d3-91bc-203a12f90c3f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.661490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 813/1609] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:42.664650 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5d749-29fc-4100-92c1-20647a42e1fd None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.665600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5d749-29fc-4100-92c1-20647a42e1fd None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.665988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5d749-29fc-4100-92c1-20647a42e1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.685664 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-32e5d749-29fc-4100-92c1-20647a42e1fd None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:42.686903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-32e5d749-29fc-4100-92c1-20647a42e1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:42.690561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:42.695116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 814/1610] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:42.696010 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5d749-29fc-4100-92c1-20647a42e1fd None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.709287 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e5d749-29fc-4100-92c1-20647a42e1fd None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.715742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da5431-ba82-4e08-a976-4f4f94c646be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.716153 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da5431-ba82-4e08-a976-4f4f94c646be admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.716534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da5431-ba82-4e08-a976-4f4f94c646be admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.716856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da5431-ba82-4e08-a976-4f4f94c646be admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.717563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da5431-ba82-4e08-a976-4f4f94c646be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.719014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 797/1611] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:42 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:42.719338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da5431-ba82-4e08-a976-4f4f94c646be admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.719918 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c509c-4698-432f-b681-a1c3ef3b7e0f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.730889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 815/1612] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:42.732505 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c509c-4698-432f-b681-a1c3ef3b7e0f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.735981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1feeb-ee07-4c53-80e0-e79be050c8bb None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.736276 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1feeb-ee07-4c53-80e0-e79be050c8bb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.736556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1feeb-ee07-4c53-80e0-e79be050c8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.739599 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de2725d-7795-4f39-8139-f290fb873d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.741579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2725d-7795-4f39-8139-f290fb873d91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.742803 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2725d-7795-4f39-8139-f290fb873d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.743970 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de2725d-7795-4f39-8139-f290fb873d91 admin admin] PATH_INFO: `/v3/projects/76e4caaf5b9e49678bc6427b71560f19/users/ca339e26746a4feaaa25b14be59479a4/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.763646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2725d-7795-4f39-8139-f290fb873d91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76e4caaf5b9e49678bc6427b71560f19, user_id=ca339e26746a4feaaa25b14be59479a4, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.766414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de2725d-7795-4f39-8139-f290fb873d91 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.779302 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-26e1feeb-ee07-4c53-80e0-e79be050c8bb None None] MFA Rules not processed for user `70b000c2b8f741eeb8199a98f939acbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:42.781500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-26e1feeb-ee07-4c53-80e0-e79be050c8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:42.786343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 798/1613] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:42 2022] PUT /identity/v3/projects/76e4caaf5b9e49678bc6427b71560f19/users/ca339e26746a4feaaa25b14be59479a4/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:42.786343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89452591-b37f-4240-a3d9-908164b03487 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.786343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89452591-b37f-4240-a3d9-908164b03487 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.786343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89452591-b37f-4240-a3d9-908164b03487 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.786343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 799/1614] 10.222.0.61 () {54 vars in 961 bytes} [Thu Jul 14 10:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:13:42.787274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89452591-b37f-4240-a3d9-908164b03487 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.867259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89452591-b37f-4240-a3d9-908164b03487 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:42.871239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 816/1615] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:42.872428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1feeb-ee07-4c53-80e0-e79be050c8bb None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.873891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a553a-8916-4260-bba4-9e242db73b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:42.874876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a553a-8916-4260-bba4-9e242db73b39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:42.875227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a553a-8916-4260-bba4-9e242db73b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:42.875571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9a553a-8916-4260-bba4-9e242db73b39 admin admin] PATH_INFO: `/v3/projects/a4cde0820dc5474eb7091fb3a8de4cd0/users/01a9eebc1c8540daa31109cfc86d5ed1/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:42.876156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-26e1feeb-ee07-4c53-80e0-e79be050c8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:42.893848 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a553a-8916-4260-bba4-9e242db73b39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4cde0820dc5474eb7091fb3a8de4cd0, user_id=01a9eebc1c8540daa31109cfc86d5ed1, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:42.898722 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9a553a-8916-4260-bba4-9e242db73b39 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:42.922029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 800/1616] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:42 2022] PUT /identity/v3/projects/a4cde0820dc5474eb7091fb3a8de4cd0/users/01a9eebc1c8540daa31109cfc86d5ed1/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:42.922029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a553a-8916-4260-bba4-9e242db73b39 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:42.963251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1feeb-ee07-4c53-80e0-e79be050c8bb None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:43.012333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9a553a-8916-4260-bba4-9e242db73b39 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:43.018757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512f9690-dde1-459b-b8be-e64fc4241e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:43.019786 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f9690-dde1-459b-b8be-e64fc4241e63 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.020327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f9690-dde1-459b-b8be-e64fc4241e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.020835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512f9690-dde1-459b-b8be-e64fc4241e63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.021840 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f9690-dde1-459b-b8be-e64fc4241e63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:43.022159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaa0f2-9b02-4a99-8183-eeca945a781c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:43.023406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaa0f2-9b02-4a99-8183-eeca945a781c service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.023679 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaa0f2-9b02-4a99-8183-eeca945a781c service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.024066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52eaa0f2-9b02-4a99-8183-eeca945a781c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.025978 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512f9690-dde1-459b-b8be-e64fc4241e63 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:43.036446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eaa0f2-9b02-4a99-8183-eeca945a781c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:43.038208 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52eaa0f2-9b02-4a99-8183-eeca945a781c service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:43.054401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 817/1617] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:13:43.056065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaa0f2-9b02-4a99-8183-eeca945a781c service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:43.059731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 801/1618] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:13:42 2022] POST /identity/v3/projects => generated 365 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:43.061164 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbf11b-1167-4739-967b-785aeb917f77 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.061451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbf11b-1167-4739-967b-785aeb917f77 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.061723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbf11b-1167-4739-967b-785aeb917f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.062704 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:43.070103 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52eaa0f2-9b02-4a99-8183-eeca945a781c service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:43.082111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-30bbf11b-1167-4739-967b-785aeb917f77 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:43.083341 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-30bbf11b-1167-4739-967b-785aeb917f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:43.115901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:43.142828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbedc8f-4ccd-4d5f-b989-67f72d3b9cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:43.144033 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbedc8f-4ccd-4d5f-b989-67f72d3b9cb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.144360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbedc8f-4ccd-4d5f-b989-67f72d3b9cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.144721 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbedc8f-4ccd-4d5f-b989-67f72d3b9cb1 admin admin] PATH_INFO: `/v3/projects/1c8ba47db4ef4546944bc305b7e580d5/users/027695398c144b12aac9aeebca663c44/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.146934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbedc8f-4ccd-4d5f-b989-67f72d3b9cb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c8ba47db4ef4546944bc305b7e580d5, user_id=027695398c144b12aac9aeebca663c44, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:43.153685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 802/1619] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:43.153859 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbedc8f-4ccd-4d5f-b989-67f72d3b9cb1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:43.154297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e37f24-9287-4e5a-922b-502544300d25 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.154297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e37f24-9287-4e5a-922b-502544300d25 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.154297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e37f24-9287-4e5a-922b-502544300d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.165413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 818/1620] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:43 2022] PUT /identity/v3/projects/1c8ba47db4ef4546944bc305b7e580d5/users/027695398c144b12aac9aeebca663c44/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:43.166231 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbedc8f-4ccd-4d5f-b989-67f72d3b9cb1 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:43.198882 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-69e37f24-9287-4e5a-922b-502544300d25 None None] MFA Rules not processed for user `e9f4efa3e2f343c2ac858a277601a7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:43.200531 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-69e37f24-9287-4e5a-922b-502544300d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:43.250178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbedc8f-4ccd-4d5f-b989-67f72d3b9cb1 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:43.255288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efb4820-f943-4f06-b447-74cefa2ccecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:43.256042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb4820-f943-4f06-b447-74cefa2ccecb admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.256394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb4820-f943-4f06-b447-74cefa2ccecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.256742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efb4820-f943-4f06-b447-74cefa2ccecb admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.258074 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb4820-f943-4f06-b447-74cefa2ccecb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:43.260192 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efb4820-f943-4f06-b447-74cefa2ccecb admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:43.297508 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 819/1621] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:13:43 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:43.297508 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd129d-9968-4696-b627-460ed441ec87 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.297508 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd129d-9968-4696-b627-460ed441ec87 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.297508 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd129d-9968-4696-b627-460ed441ec87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.301080 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 803/1622] 10.222.0.61 () {58 vars in 1058 bytes} [Thu Jul 14 10:13:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:43.303417 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e37f24-9287-4e5a-922b-502544300d25 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:43.315535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a6bd129d-9968-4696-b627-460ed441ec87 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:43.316791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a6bd129d-9968-4696-b627-460ed441ec87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:43.317899 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e37f24-9287-4e5a-922b-502544300d25 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:43.385102 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a8ea11-ffa0-4bd4-9ec8-c290ab37113e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:43.385902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8ea11-ffa0-4bd4-9ec8-c290ab37113e admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.386298 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8ea11-ffa0-4bd4-9ec8-c290ab37113e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.386731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a8ea11-ffa0-4bd4-9ec8-c290ab37113e admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.388094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8ea11-ffa0-4bd4-9ec8-c290ab37113e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:43.390312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a8ea11-ffa0-4bd4-9ec8-c290ab37113e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:43.404388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 820/1623] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:43.405296 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd129d-9968-4696-b627-460ed441ec87 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:43.422391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 804/1624] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:13:43 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:43.422391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bf23f-0cce-447a-bd3e-a1cbca60f62a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.422391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bf23f-0cce-447a-bd3e-a1cbca60f62a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.422825 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217bf23f-0cce-447a-bd3e-a1cbca60f62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.435644 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:43.471939 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd129d-9968-4696-b627-460ed441ec87 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:43.475184 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-217bf23f-0cce-447a-bd3e-a1cbca60f62a None None] MFA Rules not processed for user `027695398c144b12aac9aeebca663c44`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:43.476202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16128a-8f34-4486-8521-e562573473a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:43.476797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-217bf23f-0cce-447a-bd3e-a1cbca60f62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:43.477213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16128a-8f34-4486-8521-e562573473a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.477593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16128a-8f34-4486-8521-e562573473a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.477967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df16128a-8f34-4486-8521-e562573473a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.493112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16128a-8f34-4486-8521-e562573473a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:43.493958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df16128a-8f34-4486-8521-e562573473a6 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:43.509850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 821/1625] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:13:43.513684 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16128a-8f34-4486-8521-e562573473a6 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:43.523544 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df16128a-8f34-4486-8521-e562573473a6 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:43.530059 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6b06c-8a31-4f7c-a5bc-1d8321594158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:43.530876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6b06c-8a31-4f7c-a5bc-1d8321594158 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.531256 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6b06c-8a31-4f7c-a5bc-1d8321594158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.531661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6b06c-8a31-4f7c-a5bc-1d8321594158 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.533077 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6b06c-8a31-4f7c-a5bc-1d8321594158 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:43.537300 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6b06c-8a31-4f7c-a5bc-1d8321594158 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:43.550628 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:43.571389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 822/1626] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:13:43 2022] POST /identity/v3/users => generated 297 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:43.572431 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6b06c-8a31-4f7c-a5bc-1d8321594158 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:43.623899 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 805/1627] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:43 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:43.625345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217bf23f-0cce-447a-bd3e-a1cbca60f62a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:43.644683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217bf23f-0cce-447a-bd3e-a1cbca60f62a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:43.651714 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f829ce-77cc-4a82-bb56-e0c284cbcb5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:43.655709 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f829ce-77cc-4a82-bb56-e0c284cbcb5e service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.655995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f829ce-77cc-4a82-bb56-e0c284cbcb5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.656278 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f829ce-77cc-4a82-bb56-e0c284cbcb5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.670560 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f829ce-77cc-4a82-bb56-e0c284cbcb5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:43.672603 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f829ce-77cc-4a82-bb56-e0c284cbcb5e service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:43.679381 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6b06c-8a31-4f7c-a5bc-1d8321594158 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:43.685832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 806/1628] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:13:43.686804 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f829ce-77cc-4a82-bb56-e0c284cbcb5e service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:43.696894 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314fba1-c080-4bd1-96ef-af8bd251c98b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:43.697620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f829ce-77cc-4a82-bb56-e0c284cbcb5e service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:43.698883 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314fba1-c080-4bd1-96ef-af8bd251c98b service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.700696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314fba1-c080-4bd1-96ef-af8bd251c98b service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.701001 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5314fba1-c080-4bd1-96ef-af8bd251c98b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.704317 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed567c7-2ff8-4eae-9a5b-3bd7ce15b616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:43.705391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed567c7-2ff8-4eae-9a5b-3bd7ce15b616 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.705911 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed567c7-2ff8-4eae-9a5b-3bd7ce15b616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.706368 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed567c7-2ff8-4eae-9a5b-3bd7ce15b616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.707534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed567c7-2ff8-4eae-9a5b-3bd7ce15b616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:43.710795 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed567c7-2ff8-4eae-9a5b-3bd7ce15b616 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:43.738387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 807/1629] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:43.740713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed567c7-2ff8-4eae-9a5b-3bd7ce15b616 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:43.768428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed567c7-2ff8-4eae-9a5b-3bd7ce15b616 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:43.780371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321db71-b4e2-47a8-94aa-6d9f5facb959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:43.781127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321db71-b4e2-47a8-94aa-6d9f5facb959 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.783688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321db71-b4e2-47a8-94aa-6d9f5facb959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.783981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a321db71-b4e2-47a8-94aa-6d9f5facb959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.784684 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314fba1-c080-4bd1-96ef-af8bd251c98b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:43.784858 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a321db71-b4e2-47a8-94aa-6d9f5facb959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:43.786627 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5314fba1-c080-4bd1-96ef-af8bd251c98b service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:43.791477 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a321db71-b4e2-47a8-94aa-6d9f5facb959 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:43.799128 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 823/1630] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:13:43.800159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314fba1-c080-4bd1-96ef-af8bd251c98b service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:43.828733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 808/1631] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:43.829850 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321db71-b4e2-47a8-94aa-6d9f5facb959 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:43.855184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:43.857819 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a321db71-b4e2-47a8-94aa-6d9f5facb959 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:43.866645 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5314fba1-c080-4bd1-96ef-af8bd251c98b service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:43.867602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9db21-3c62-4e02-bc92-d7ddeae67493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:43.868526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9db21-3c62-4e02-bc92-d7ddeae67493 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.868814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9db21-3c62-4e02-bc92-d7ddeae67493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.869126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9db21-3c62-4e02-bc92-d7ddeae67493 admin admin] PATH_INFO: `/v3/projects/3adeb06f835f41dcb90e36b1819fc86e/users/3c0804f1161c4b66b96bcc406e86c72f/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.871537 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea2e57-918c-47e3-b6b7-8030c18e85b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:43.872365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea2e57-918c-47e3-b6b7-8030c18e85b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.872738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea2e57-918c-47e3-b6b7-8030c18e85b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.873360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea2e57-918c-47e3-b6b7-8030c18e85b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.886069 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea2e57-918c-47e3-b6b7-8030c18e85b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:43.888064 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea2e57-918c-47e3-b6b7-8030c18e85b4 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:43.900759 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 824/1632] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:43 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:13:43.901694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea2e57-918c-47e3-b6b7-8030c18e85b4 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:43.919641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9db21-3c62-4e02-bc92-d7ddeae67493 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3adeb06f835f41dcb90e36b1819fc86e, user_id=3c0804f1161c4b66b96bcc406e86c72f, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:43.923219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea2e57-918c-47e3-b6b7-8030c18e85b4 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:43.923707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9db21-3c62-4e02-bc92-d7ddeae67493 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:43.932217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea762be-8961-42c3-951e-94f8ece28df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:43.936161 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea762be-8961-42c3-951e-94f8ece28df7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:43.936454 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea762be-8961-42c3-951e-94f8ece28df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:43.937225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea762be-8961-42c3-951e-94f8ece28df7 admin admin] PATH_INFO: `/v3/projects/642f2f01d3814a40be2b7bf16a584486/users/2e3043d3515c41b88b3bc8ecdeb68176/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:43.964194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 809/1633] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:43 2022] PUT /identity/v3/projects/3adeb06f835f41dcb90e36b1819fc86e/users/3c0804f1161c4b66b96bcc406e86c72f/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:43.966063 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9db21-3c62-4e02-bc92-d7ddeae67493 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:43.972146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea762be-8961-42c3-951e-94f8ece28df7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=642f2f01d3814a40be2b7bf16a584486, user_id=2e3043d3515c41b88b3bc8ecdeb68176, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:43.976582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea762be-8961-42c3-951e-94f8ece28df7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:44.015896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 825/1634] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:43 2022] PUT /identity/v3/projects/642f2f01d3814a40be2b7bf16a584486/users/2e3043d3515c41b88b3bc8ecdeb68176/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:44.020133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea762be-8961-42c3-951e-94f8ece28df7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:44.091217 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9db21-3c62-4e02-bc92-d7ddeae67493 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:44.202412 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea762be-8961-42c3-951e-94f8ece28df7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:44.211217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106dd37-23be-4809-bfe4-8ec5f71dc7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:44.211695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ea217-9ab0-492a-9f18-95c535aecd71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:44.212071 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106dd37-23be-4809-bfe4-8ec5f71dc7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:44.212422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106dd37-23be-4809-bfe4-8ec5f71dc7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:44.212779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d106dd37-23be-4809-bfe4-8ec5f71dc7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:44.213662 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ea217-9ab0-492a-9f18-95c535aecd71 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:44.213828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106dd37-23be-4809-bfe4-8ec5f71dc7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:44.213990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ea217-9ab0-492a-9f18-95c535aecd71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:44.214141 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ea217-9ab0-492a-9f18-95c535aecd71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:44.217073 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ea217-9ab0-492a-9f18-95c535aecd71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:44.218348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d106dd37-23be-4809-bfe4-8ec5f71dc7d9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:44.220142 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ea217-9ab0-492a-9f18-95c535aecd71 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:44.259694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 810/1635] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:44.264703 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ea217-9ab0-492a-9f18-95c535aecd71 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:44.299822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 826/1636] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:44.301468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106dd37-23be-4809-bfe4-8ec5f71dc7d9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:44.301855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ea217-9ab0-492a-9f18-95c535aecd71 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:44.321218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8b6f4-8d1a-43d8-8c71-a73cad51ef96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:44.326750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d106dd37-23be-4809-bfe4-8ec5f71dc7d9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:44.327370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8b6f4-8d1a-43d8-8c71-a73cad51ef96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:44.327865 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8b6f4-8d1a-43d8-8c71-a73cad51ef96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:44.329305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8b6f4-8d1a-43d8-8c71-a73cad51ef96 admin admin] PATH_INFO: `/v3/projects/3adeb06f835f41dcb90e36b1819fc86e/users/3c0804f1161c4b66b96bcc406e86c72f/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:44.340883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8b6f4-8d1a-43d8-8c71-a73cad51ef96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3adeb06f835f41dcb90e36b1819fc86e, user_id=3c0804f1161c4b66b96bcc406e86c72f, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:44.344633 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e4c652-4b3e-4514-847c-03711ad682fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:44.345603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4c652-4b3e-4514-847c-03711ad682fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:44.345866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4c652-4b3e-4514-847c-03711ad682fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:44.347824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e4c652-4b3e-4514-847c-03711ad682fb admin admin] PATH_INFO: `/v3/projects/642f2f01d3814a40be2b7bf16a584486/users/2e3043d3515c41b88b3bc8ecdeb68176/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:44.352652 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4c652-4b3e-4514-847c-03711ad682fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=642f2f01d3814a40be2b7bf16a584486, user_id=2e3043d3515c41b88b3bc8ecdeb68176, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:44.353720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8b6f4-8d1a-43d8-8c71-a73cad51ef96 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:44.355304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e4c652-4b3e-4514-847c-03711ad682fb admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:44.378156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:44.394120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 811/1637] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:44 2022] PUT /identity/v3/projects/3adeb06f835f41dcb90e36b1819fc86e/users/3c0804f1161c4b66b96bcc406e86c72f/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:13:44.405785 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48672b70-6693-459e-94cb-04e84f67fceb None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:44.406435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48672b70-6693-459e-94cb-04e84f67fceb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:44.406682 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48672b70-6693-459e-94cb-04e84f67fceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:44.418983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 827/1638] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:44 2022] PUT /identity/v3/projects/642f2f01d3814a40be2b7bf16a584486/users/2e3043d3515c41b88b3bc8ecdeb68176/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:44.422037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3a635-fc37-4550-b073-51e13f2f4fea None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:44.422318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3a635-fc37-4550-b073-51e13f2f4fea None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:44.422590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e3a635-fc37-4550-b073-51e13f2f4fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:44.508286 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-48672b70-6693-459e-94cb-04e84f67fceb None None] MFA Rules not processed for user `3c0804f1161c4b66b96bcc406e86c72f`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:44.511041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-48672b70-6693-459e-94cb-04e84f67fceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:44.521884 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-37e3a635-fc37-4550-b073-51e13f2f4fea None None] MFA Rules not processed for user `2e3043d3515c41b88b3bc8ecdeb68176`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:44.525200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-37e3a635-fc37-4550-b073-51e13f2f4fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:44.690074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 812/1639] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:13:44.691771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdc8b4-076d-4ce0-a1e8-be341f28ea2d None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:44.692166 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdc8b4-076d-4ce0-a1e8-be341f28ea2d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:44.692550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdc8b4-076d-4ce0-a1e8-be341f28ea2d None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:44.695071 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 813/1640] 10.222.0.61 () {54 vars in 912 bytes} [Thu Jul 14 10:13:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:13:44.700088 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdc8b4-076d-4ce0-a1e8-be341f28ea2d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:44.740471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 828/1641] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:44 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:44.745840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb436d9c-d25a-4be3-8daa-400ca997b89d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:44.746349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb436d9c-d25a-4be3-8daa-400ca997b89d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:44.747323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb436d9c-d25a-4be3-8daa-400ca997b89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:44.795425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-cb436d9c-d25a-4be3-8daa-400ca997b89d None None] MFA Rules not processed for user `e9f4efa3e2f343c2ac858a277601a7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:44.796783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-cb436d9c-d25a-4be3-8daa-400ca997b89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:44.841613 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdc8b4-076d-4ce0-a1e8-be341f28ea2d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:44.849120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83437f5e-4c57-4a66-8a17-511633453335 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:44.849898 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83437f5e-4c57-4a66-8a17-511633453335 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:44.850177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83437f5e-4c57-4a66-8a17-511633453335 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:44.850450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83437f5e-4c57-4a66-8a17-511633453335 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:44.860815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:44.869598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83437f5e-4c57-4a66-8a17-511633453335 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:44.871099 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83437f5e-4c57-4a66-8a17-511633453335 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:44.885934 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 814/1642] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:13:44.887041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83437f5e-4c57-4a66-8a17-511633453335 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:44.899417 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83437f5e-4c57-4a66-8a17-511633453335 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:44.904631 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6372f6e-ec4a-4323-ac6d-2a60ff84191b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:44.905563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6372f6e-ec4a-4323-ac6d-2a60ff84191b service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:44.906837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6372f6e-ec4a-4323-ac6d-2a60ff84191b service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:44.906837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6372f6e-ec4a-4323-ac6d-2a60ff84191b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:44.923430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6372f6e-ec4a-4323-ac6d-2a60ff84191b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:44.927600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6372f6e-ec4a-4323-ac6d-2a60ff84191b service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:44.946958 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 815/1643] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:13:44.966060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 829/1644] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Jul 14 10:13:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:45.007589 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6372f6e-ec4a-4323-ac6d-2a60ff84191b service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:45.018211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6372f6e-ec4a-4323-ac6d-2a60ff84191b service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:45.029657 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d83522-8781-472d-8082-fa548cc280ae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:45.036098 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d83522-8781-472d-8082-fa548cc280ae service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:45.036386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d83522-8781-472d-8082-fa548cc280ae service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:45.037821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d83522-8781-472d-8082-fa548cc280ae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:45.061568 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d83522-8781-472d-8082-fa548cc280ae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:45.067069 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d83522-8781-472d-8082-fa548cc280ae service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:45.097488 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 816/1645] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:13:45.236319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb436d9c-d25a-4be3-8daa-400ca997b89d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:45.255733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb436d9c-d25a-4be3-8daa-400ca997b89d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:45.266372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714755f-6cbb-42ea-b78f-ca358f68a0ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:45.267507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714755f-6cbb-42ea-b78f-ca358f68a0ad service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:45.267959 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714755f-6cbb-42ea-b78f-ca358f68a0ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:45.268946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714755f-6cbb-42ea-b78f-ca358f68a0ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:45.460906 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714755f-6cbb-42ea-b78f-ca358f68a0ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:45.462415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714755f-6cbb-42ea-b78f-ca358f68a0ad service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:45.565562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 830/1646] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:45 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:13:45.894266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d83522-8781-472d-8082-fa548cc280ae service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:45.911939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d83522-8781-472d-8082-fa548cc280ae service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:45.916412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbb73c8-9680-466c-8053-082224a1345a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:45.917094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb73c8-9680-466c-8053-082224a1345a service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:45.917372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb73c8-9680-466c-8053-082224a1345a service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:45.917640 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbb73c8-9680-466c-8053-082224a1345a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:46.068222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb73c8-9680-466c-8053-082224a1345a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:46.070863 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbb73c8-9680-466c-8053-082224a1345a service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:46.076104 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:46.082090 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 817/1647] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:13:46.550511 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714755f-6cbb-42ea-b78f-ca358f68a0ad service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:46.693264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714755f-6cbb-42ea-b78f-ca358f68a0ad service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:46.699404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1a116-3318-4649-80b5-3c3632a8dacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:46.699925 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1a116-3318-4649-80b5-3c3632a8dacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:46.700269 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1a116-3318-4649-80b5-3c3632a8dacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:46.700607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1a116-3318-4649-80b5-3c3632a8dacb admin admin] PATH_INFO: `/v3/users/23dad834c85e4246b00c3b232b5193e2` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:46.702629 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1a116-3318-4649-80b5-3c3632a8dacb admin admin] RBAC: Authorizing `identity:delete_user(user_id=23dad834c85e4246b00c3b232b5193e2)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:46.706439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: WARNING py.warnings [None req-7fd1a116-3318-4649-80b5-3c3632a8dacb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:13:46.706439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: warnings.warn(msg) Jul 14 10:13:46.706439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: Jul 14 10:13:46.707880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd1a116-3318-4649-80b5-3c3632a8dacb admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:46.834358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7fd1a116-3318-4649-80b5-3c3632a8dacb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23dad834c85e4246b00c3b232b5193e2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:46.836298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7fd1a116-3318-4649-80b5-3c3632a8dacb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23dad834c85e4246b00c3b232b5193e2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:46.897585 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:46.900827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7fd1a116-3318-4649-80b5-3c3632a8dacb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23dad834c85e4246b00c3b232b5193e2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:46.927054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7fd1a116-3318-4649-80b5-3c3632a8dacb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23dad834c85e4246b00c3b232b5193e2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:46.941812 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7fd1a116-3318-4649-80b5-3c3632a8dacb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23dad834c85e4246b00c3b232b5193e2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:46.955722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 831/1648] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:13:46 2022] DELETE /identity/v3/users/23dad834c85e4246b00c3b232b5193e2 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:46.957867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3dafa-1b15-4c48-8ffc-e9d64d75d0b7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:46.958856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3dafa-1b15-4c48-8ffc-e9d64d75d0b7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:46.959529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a3dafa-1b15-4c48-8ffc-e9d64d75d0b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:47.008155 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f9a3dafa-1b15-4c48-8ffc-e9d64d75d0b7 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:47.009630 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f9a3dafa-1b15-4c48-8ffc-e9d64d75d0b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:47.162766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 818/1649] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:47.187177 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1a116-3318-4649-80b5-3c3632a8dacb admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:47.189957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7fd1a116-3318-4649-80b5-3c3632a8dacb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:47.344545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1a116-3318-4649-80b5-3c3632a8dacb admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:47.354683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc129df-393e-4f50-8c4e-8cdf13eb1d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:47.356887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc129df-393e-4f50-8c4e-8cdf13eb1d5e service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:47.358479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc129df-393e-4f50-8c4e-8cdf13eb1d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:47.358854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc129df-393e-4f50-8c4e-8cdf13eb1d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:47.386415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc129df-393e-4f50-8c4e-8cdf13eb1d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:47.388149 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc129df-393e-4f50-8c4e-8cdf13eb1d5e service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:47.412019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 832/1650] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:13:47.998147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe18af1-8875-4dc3-8d85-7bbb7c833c4a None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:47.998669 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe18af1-8875-4dc3-8d85-7bbb7c833c4a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:47.998669 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe18af1-8875-4dc3-8d85-7bbb7c833c4a None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:48.001076 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 819/1651] 10.222.0.61 () {54 vars in 961 bytes} [Thu Jul 14 10:13:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:13:48.013686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad84f3-a7c5-4e21-93dc-a79624771c11 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:48.014184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad84f3-a7c5-4e21-93dc-a79624771c11 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:48.014266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ad84f3-a7c5-4e21-93dc-a79624771c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:48.064630 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-96ad84f3-a7c5-4e21-93dc-a79624771c11 None None] MFA Rules not processed for user `e9f4efa3e2f343c2ac858a277601a7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:48.066066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-96ad84f3-a7c5-4e21-93dc-a79624771c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:48.172732 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2aaccd-8a5f-4b39-b1c1-39ec21112758 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:48.173566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2aaccd-8a5f-4b39-b1c1-39ec21112758 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:48.173999 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2aaccd-8a5f-4b39-b1c1-39ec21112758 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:48.180032 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 820/1652] 10.222.0.61 () {54 vars in 912 bytes} [Thu Jul 14 10:13:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:13:48.192516 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb7818-8ad8-46aa-b230-7de1e050c986 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:48.192804 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb7818-8ad8-46aa-b230-7de1e050c986 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:48.193107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb7818-8ad8-46aa-b230-7de1e050c986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:48.195039 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 833/1653] 10.222.0.61 () {58 vars in 1058 bytes} [Thu Jul 14 10:13:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:48.214666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-addb7818-8ad8-46aa-b230-7de1e050c986 None None] MFA Rules not processed for user `e9f4efa3e2f343c2ac858a277601a7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:48.215911 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-addb7818-8ad8-46aa-b230-7de1e050c986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:48.253972 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 821/1654] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Jul 14 10:13:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:48.270810 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad84f3-a7c5-4e21-93dc-a79624771c11 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:48.273341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-96ad84f3-a7c5-4e21-93dc-a79624771c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:48.363841 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ad84f3-a7c5-4e21-93dc-a79624771c11 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:48.368407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16b0a5-309b-456f-bf1f-2efe14d7c9d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:48.370072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16b0a5-309b-456f-bf1f-2efe14d7c9d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:48.370526 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16b0a5-309b-456f-bf1f-2efe14d7c9d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:48.370832 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16b0a5-309b-456f-bf1f-2efe14d7c9d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:48.404321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16b0a5-309b-456f-bf1f-2efe14d7c9d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:48.407220 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f16b0a5-309b-456f-bf1f-2efe14d7c9d4 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:48.434529 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 834/1655] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:13:49.420470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb7818-8ad8-46aa-b230-7de1e050c986 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:49.422786 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-addb7818-8ad8-46aa-b230-7de1e050c986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:49.474401 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb7818-8ad8-46aa-b230-7de1e050c986 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:49.498183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:49.500909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:49.502312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:49.502312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] PATH_INFO: `/v3/projects/c3aace1f1bdb4d809577f16539279436` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:49.503570 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3aace1f1bdb4d809577f16539279436)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:49.506128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: WARNING py.warnings [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 14 10:13:49.506128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: warnings.warn(msg) Jul 14 10:13:49.506128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: Jul 14 10:13:49.516544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:49.634649 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3aace1f1bdb4d809577f16539279436', 'request_id': 'req-f81af347-eabd-43eb-b4f7-941d677afd55'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:49.657008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3aace1f1bdb4d809577f16539279436', 'request_id': 'req-f81af347-eabd-43eb-b4f7-941d677afd55'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:49.676597 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3aace1f1bdb4d809577f16539279436', 'request_id': 'req-f81af347-eabd-43eb-b4f7-941d677afd55'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:49.720279 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] The token cache is being invalidate because project c3aace1f1bdb4d809577f16539279436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:13:49.721545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:49.723235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:49.745872 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16b0a5-309b-456f-bf1f-2efe14d7c9d4 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:49.746336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 822/1656] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:13:49 2022] DELETE /identity/v3/projects/c3aace1f1bdb4d809577f16539279436 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:49.763816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16b0a5-309b-456f-bf1f-2efe14d7c9d4 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:49.794079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:49.905592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5464b-6225-4411-bf92-8747027ecaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:49.907972 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5464b-6225-4411-bf92-8747027ecaf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:49.908473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5464b-6225-4411-bf92-8747027ecaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:49.909013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5464b-6225-4411-bf92-8747027ecaf7 admin admin] PATH_INFO: `/v3/users/027695398c144b12aac9aeebca663c44` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:49.910528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5464b-6225-4411-bf92-8747027ecaf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=027695398c144b12aac9aeebca663c44)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:49.913714 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb5464b-6225-4411-bf92-8747027ecaf7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:50.033378 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fdb5464b-6225-4411-bf92-8747027ecaf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '027695398c144b12aac9aeebca663c44'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:50.034408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fdb5464b-6225-4411-bf92-8747027ecaf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '027695398c144b12aac9aeebca663c44'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:50.070743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fdb5464b-6225-4411-bf92-8747027ecaf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '027695398c144b12aac9aeebca663c44'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:50.082680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fdb5464b-6225-4411-bf92-8747027ecaf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '027695398c144b12aac9aeebca663c44'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:50.105587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fdb5464b-6225-4411-bf92-8747027ecaf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '027695398c144b12aac9aeebca663c44'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:50.117805 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 835/1657] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:13:49 2022] DELETE /identity/v3/users/027695398c144b12aac9aeebca663c44 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:50.148452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:50.150188 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:50.188676 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81af347-eabd-43eb-b4f7-941d677afd55 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:50.325572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e682a-60dd-4b5a-86ef-9a56738cdb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:50.326705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e682a-60dd-4b5a-86ef-9a56738cdb2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:50.327054 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e682a-60dd-4b5a-86ef-9a56738cdb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:50.327400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e682a-60dd-4b5a-86ef-9a56738cdb2d admin admin] PATH_INFO: `/v3/users/b8e0c072b7a24d30893957523763e851` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:50.328823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e682a-60dd-4b5a-86ef-9a56738cdb2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8e0c072b7a24d30893957523763e851)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:50.330872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e682a-60dd-4b5a-86ef-9a56738cdb2d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:50.364010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:50.414701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b28e682a-60dd-4b5a-86ef-9a56738cdb2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8e0c072b7a24d30893957523763e851'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:50.416139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b28e682a-60dd-4b5a-86ef-9a56738cdb2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8e0c072b7a24d30893957523763e851'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:50.436468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b28e682a-60dd-4b5a-86ef-9a56738cdb2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8e0c072b7a24d30893957523763e851'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:50.450167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b28e682a-60dd-4b5a-86ef-9a56738cdb2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8e0c072b7a24d30893957523763e851'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:50.478685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b28e682a-60dd-4b5a-86ef-9a56738cdb2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8e0c072b7a24d30893957523763e851'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:50.500295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 823/1658] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:13:50 2022] DELETE /identity/v3/users/b8e0c072b7a24d30893957523763e851 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:50.501482 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac4d94-8a2c-4909-a2a7-b920558dc28a None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:50.501482 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac4d94-8a2c-4909-a2a7-b920558dc28a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:50.501684 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ac4d94-8a2c-4909-a2a7-b920558dc28a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:50.521617 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-13ac4d94-8a2c-4909-a2a7-b920558dc28a None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:50.524201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-13ac4d94-8a2c-4909-a2a7-b920558dc28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:50.647108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 836/1659] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:50.683338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e682a-60dd-4b5a-86ef-9a56738cdb2d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:50.685014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b28e682a-60dd-4b5a-86ef-9a56738cdb2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:50.822061 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e682a-60dd-4b5a-86ef-9a56738cdb2d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:50.833194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe1505-c77a-429b-9feb-85966795cfba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:50.834008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe1505-c77a-429b-9feb-85966795cfba service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:50.834404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe1505-c77a-429b-9feb-85966795cfba service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:50.834614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe1505-c77a-429b-9feb-85966795cfba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:50.857122 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe1505-c77a-429b-9feb-85966795cfba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:50.860882 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe1505-c77a-429b-9feb-85966795cfba service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:50.881238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 824/1660] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:13:53.071225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac4d94-8a2c-4909-a2a7-b920558dc28a None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:53.073107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-13ac4d94-8a2c-4909-a2a7-b920558dc28a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:53.113239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ac4d94-8a2c-4909-a2a7-b920558dc28a None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:53.132079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c45f1-bcea-4fd7-9cc6-cc16a4ddc5b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:53.133301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c45f1-bcea-4fd7-9cc6-cc16a4ddc5b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:53.135318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c45f1-bcea-4fd7-9cc6-cc16a4ddc5b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:53.135781 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c45f1-bcea-4fd7-9cc6-cc16a4ddc5b4 admin admin] PATH_INFO: `/v3/projects/1c8ba47db4ef4546944bc305b7e580d5` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:53.143074 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c45f1-bcea-4fd7-9cc6-cc16a4ddc5b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c8ba47db4ef4546944bc305b7e580d5)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:53.155356 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7c45f1-bcea-4fd7-9cc6-cc16a4ddc5b4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:53.251996 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:53.278138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb7c45f1-bcea-4fd7-9cc6-cc16a4ddc5b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c8ba47db4ef4546944bc305b7e580d5', 'request_id': 'req-bb7c45f1-bcea-4fd7-9cc6-cc16a4ddc5b4'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:53.290678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb7c45f1-bcea-4fd7-9cc6-cc16a4ddc5b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c8ba47db4ef4546944bc305b7e580d5', 'request_id': 'req-bb7c45f1-bcea-4fd7-9cc6-cc16a4ddc5b4'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:53.299887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb7c45f1-bcea-4fd7-9cc6-cc16a4ddc5b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c8ba47db4ef4546944bc305b7e580d5', 'request_id': 'req-bb7c45f1-bcea-4fd7-9cc6-cc16a4ddc5b4'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:53.331970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb7c45f1-bcea-4fd7-9cc6-cc16a4ddc5b4 admin admin] The token cache is being invalidate because project 1c8ba47db4ef4546944bc305b7e580d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:13:53.332541 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb7c45f1-bcea-4fd7-9cc6-cc16a4ddc5b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:53.333530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb7c45f1-bcea-4fd7-9cc6-cc16a4ddc5b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:53.336733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 837/1661] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:13:53 2022] DELETE /identity/v3/projects/1c8ba47db4ef4546944bc305b7e580d5 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:13:53.344151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea569d-fea8-494a-9c16-315e52e2bb31 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:53.344652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea569d-fea8-494a-9c16-315e52e2bb31 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:53.344727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea569d-fea8-494a-9c16-315e52e2bb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:53.389906 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e8ea569d-fea8-494a-9c16-315e52e2bb31 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:53.391567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e8ea569d-fea8-494a-9c16-315e52e2bb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:53.536814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 825/1662] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:53.545283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2485813-39db-4337-a897-1f85d30ef68e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:53.545679 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2485813-39db-4337-a897-1f85d30ef68e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:53.546447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2485813-39db-4337-a897-1f85d30ef68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:53.580903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f2485813-39db-4337-a897-1f85d30ef68e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:53.583805 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f2485813-39db-4337-a897-1f85d30ef68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:53.606071 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 838/1663] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:53.609691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea569d-fea8-494a-9c16-315e52e2bb31 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:53.626777 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea569d-fea8-494a-9c16-315e52e2bb31 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:53.635626 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c5c41-0a7a-47b4-9d01-1690b475f16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:53.636778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c5c41-0a7a-47b4-9d01-1690b475f16f admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:53.637363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c5c41-0a7a-47b4-9d01-1690b475f16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:53.637794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9c5c41-0a7a-47b4-9d01-1690b475f16f admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:53.639534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c5c41-0a7a-47b4-9d01-1690b475f16f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:13:53.639962 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c5c41-0a7a-47b4-9d01-1690b475f16f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:53.657429 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9c5c41-0a7a-47b4-9d01-1690b475f16f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:53.712575 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2485813-39db-4337-a897-1f85d30ef68e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:53.713210 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 826/1664] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:13:53.752464 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2485813-39db-4337-a897-1f85d30ef68e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:53.763095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c42c3-df09-47a1-87e0-bd849eec478e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:53.763790 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c42c3-df09-47a1-87e0-bd849eec478e admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:53.764071 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c42c3-df09-47a1-87e0-bd849eec478e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:53.764346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5c42c3-df09-47a1-87e0-bd849eec478e admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:53.766396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c42c3-df09-47a1-87e0-bd849eec478e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:53.768379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5c42c3-df09-47a1-87e0-bd849eec478e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:53.882801 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 839/1665] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:13:53 2022] POST /identity/v3/projects => generated 369 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:53.884371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c5c41-0a7a-47b4-9d01-1690b475f16f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:53.908190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9c5c41-0a7a-47b4-9d01-1690b475f16f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:53.912562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c42c3-df09-47a1-87e0-bd849eec478e admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:53.914372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0b5c42c3-df09-47a1-87e0-bd849eec478e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:53.980884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:54.077052 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f98373-b442-4200-ac2f-bd5b7313956f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:54.077404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f98373-b442-4200-ac2f-bd5b7313956f admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:54.077657 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f98373-b442-4200-ac2f-bd5b7313956f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:54.077929 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f98373-b442-4200-ac2f-bd5b7313956f admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:54.083948 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f98373-b442-4200-ac2f-bd5b7313956f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:54.088316 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5c42c3-df09-47a1-87e0-bd849eec478e admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:54.091679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f98373-b442-4200-ac2f-bd5b7313956f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:54.108393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6f9459-e50a-4f11-97eb-04b9293bd2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:54.109332 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f9459-e50a-4f11-97eb-04b9293bd2d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:54.109785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f9459-e50a-4f11-97eb-04b9293bd2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:54.109864 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6f9459-e50a-4f11-97eb-04b9293bd2d3 admin admin] PATH_INFO: `/v3/projects/a047dd397bbe42c487a4a2dce8cf6517` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:54.115962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f9459-e50a-4f11-97eb-04b9293bd2d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a047dd397bbe42c487a4a2dce8cf6517)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:54.125964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6f9459-e50a-4f11-97eb-04b9293bd2d3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:54.157735 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 827/1666] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:13:53 2022] POST /identity/v3/users => generated 285 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:54.159741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f98373-b442-4200-ac2f-bd5b7313956f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:54.192359 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f98373-b442-4200-ac2f-bd5b7313956f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:54.226567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211857d-3d46-4583-a83e-e9c6791be2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:54.227307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211857d-3d46-4583-a83e-e9c6791be2e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:54.227586 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211857d-3d46-4583-a83e-e9c6791be2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:54.227868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211857d-3d46-4583-a83e-e9c6791be2e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:54.229490 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211857d-3d46-4583-a83e-e9c6791be2e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:54.231319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211857d-3d46-4583-a83e-e9c6791be2e2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:54.273736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 828/1667] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:54.274971 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211857d-3d46-4583-a83e-e9c6791be2e2 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:54.285311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2c6f9459-e50a-4f11-97eb-04b9293bd2d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a047dd397bbe42c487a4a2dce8cf6517', 'request_id': 'req-2c6f9459-e50a-4f11-97eb-04b9293bd2d3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:54.299328 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211857d-3d46-4583-a83e-e9c6791be2e2 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:54.307118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2c6f9459-e50a-4f11-97eb-04b9293bd2d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a047dd397bbe42c487a4a2dce8cf6517', 'request_id': 'req-2c6f9459-e50a-4f11-97eb-04b9293bd2d3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:54.329643 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2c6f9459-e50a-4f11-97eb-04b9293bd2d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a047dd397bbe42c487a4a2dce8cf6517', 'request_id': 'req-2c6f9459-e50a-4f11-97eb-04b9293bd2d3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:54.358953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2c6f9459-e50a-4f11-97eb-04b9293bd2d3 admin admin] The token cache is being invalidate because project a047dd397bbe42c487a4a2dce8cf6517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:13:54.359310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2c6f9459-e50a-4f11-97eb-04b9293bd2d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:54.361324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-2c6f9459-e50a-4f11-97eb-04b9293bd2d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:13:54.363603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 840/1668] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:13:53 2022] DELETE /identity/v3/projects/a047dd397bbe42c487a4a2dce8cf6517 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:13:54.370928 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78f71b-4898-4231-8055-c10d1ca598b9 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:54.370928 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78f71b-4898-4231-8055-c10d1ca598b9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:54.370928 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c78f71b-4898-4231-8055-c10d1ca598b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:54.424236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f314b-3156-4962-b957-e92ccc9edc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:54.424954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f314b-3156-4962-b957-e92ccc9edc8f admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:54.431302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f314b-3156-4962-b957-e92ccc9edc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:54.431600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f314b-3156-4962-b957-e92ccc9edc8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:54.432409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f314b-3156-4962-b957-e92ccc9edc8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:54.438391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8c78f71b-4898-4231-8055-c10d1ca598b9 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:54.441539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8c78f71b-4898-4231-8055-c10d1ca598b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:54.444557 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f314b-3156-4962-b957-e92ccc9edc8f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:54.455552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 841/1669] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:54.463205 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca96215-6f5f-4b45-a50e-a46727875bf9 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:54.463799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca96215-6f5f-4b45-a50e-a46727875bf9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:54.464274 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca96215-6f5f-4b45-a50e-a46727875bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:54.494998 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 829/1670] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:54.498591 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f314b-3156-4962-b957-e92ccc9edc8f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:54.503400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-187f314b-3156-4962-b957-e92ccc9edc8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:54.521709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0ca96215-6f5f-4b45-a50e-a46727875bf9 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:54.526066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0ca96215-6f5f-4b45-a50e-a46727875bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:54.536495 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f314b-3156-4962-b957-e92ccc9edc8f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:54.556563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb098b8-8860-4e88-96dd-e367f3118f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:54.561269 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 842/1671] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:54.565740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca96215-6f5f-4b45-a50e-a46727875bf9 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:54.566642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb098b8-8860-4e88-96dd-e367f3118f86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:54.567015 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb098b8-8860-4e88-96dd-e367f3118f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:54.567306 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb098b8-8860-4e88-96dd-e367f3118f86 admin admin] PATH_INFO: `/v3/projects/b22f3f4500c24fe9ac26316c2f6b99ec/users/3b8a4164ad9f4948bd25d3643ccdc12b/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:54.587527 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca96215-6f5f-4b45-a50e-a46727875bf9 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:54.609222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb098b8-8860-4e88-96dd-e367f3118f86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b22f3f4500c24fe9ac26316c2f6b99ec, user_id=3b8a4164ad9f4948bd25d3643ccdc12b, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:54.609898 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac854b-6cc2-4f77-a6b7-3f597ecbf6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:54.610820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac854b-6cc2-4f77-a6b7-3f597ecbf6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:54.611969 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb098b8-8860-4e88-96dd-e367f3118f86 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:54.612357 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac854b-6cc2-4f77-a6b7-3f597ecbf6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:54.612654 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac854b-6cc2-4f77-a6b7-3f597ecbf6a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:54.617908 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac854b-6cc2-4f77-a6b7-3f597ecbf6a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:13:54.618237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac854b-6cc2-4f77-a6b7-3f597ecbf6a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:54.624869 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac854b-6cc2-4f77-a6b7-3f597ecbf6a2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:54.635062 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:54.646952 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 830/1672] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:54 2022] PUT /identity/v3/projects/b22f3f4500c24fe9ac26316c2f6b99ec/users/3b8a4164ad9f4948bd25d3643ccdc12b/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:54.663155 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a212e09-9d69-4b13-9e8c-766f20aaa9ed None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:54.664071 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a212e09-9d69-4b13-9e8c-766f20aaa9ed None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:54.664336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a212e09-9d69-4b13-9e8c-766f20aaa9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:54.670466 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 843/1673] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:13:54.678131 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac854b-6cc2-4f77-a6b7-3f597ecbf6a2 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:54.718231 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac854b-6cc2-4f77-a6b7-3f597ecbf6a2 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:54.733737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6a212e09-9d69-4b13-9e8c-766f20aaa9ed None None] MFA Rules not processed for user `3b8a4164ad9f4948bd25d3643ccdc12b`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:54.734995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6a212e09-9d69-4b13-9e8c-766f20aaa9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:54.881544 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd57a6-1bdb-4aaf-9674-364b56df6c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:54.882504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd57a6-1bdb-4aaf-9674-364b56df6c5e admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:54.882905 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd57a6-1bdb-4aaf-9674-364b56df6c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:54.883299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd57a6-1bdb-4aaf-9674-364b56df6c5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:54.884151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd57a6-1bdb-4aaf-9674-364b56df6c5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:54.888297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd57a6-1bdb-4aaf-9674-364b56df6c5e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:54.905251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 831/1674] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:54 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:13:54.910085 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a212e09-9d69-4b13-9e8c-766f20aaa9ed None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:54.927518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a212e09-9d69-4b13-9e8c-766f20aaa9ed None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:54.933820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6ca75-1968-4c56-b92a-8fd4d197f551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:54.934869 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6ca75-1968-4c56-b92a-8fd4d197f551 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:54.935538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6ca75-1968-4c56-b92a-8fd4d197f551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:54.935902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6ca75-1968-4c56-b92a-8fd4d197f551 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:54.936813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6ca75-1968-4c56-b92a-8fd4d197f551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:54.941982 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d6ca75-1968-4c56-b92a-8fd4d197f551 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:54.945351 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 844/1675] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:13:54 2022] POST /identity/v3/projects => generated 405 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:54.949694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd57a6-1bdb-4aaf-9674-364b56df6c5e admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:54.958555 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:55.023754 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 832/1676] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:13:54 2022] POST /identity/v3/projects => generated 367 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:55.028369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6ca75-1968-4c56-b92a-8fd4d197f551 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:55.116707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd57a6-1bdb-4aaf-9674-364b56df6c5e admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:55.198087 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6ca75-1968-4c56-b92a-8fd4d197f551 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:55.216259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fced4b-b471-478b-a8be-eb2c200ef7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:55.220286 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fced4b-b471-478b-a8be-eb2c200ef7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:55.220607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fced4b-b471-478b-a8be-eb2c200ef7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:55.220963 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fced4b-b471-478b-a8be-eb2c200ef7d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:55.221833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fced4b-b471-478b-a8be-eb2c200ef7d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:55.224463 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fced4b-b471-478b-a8be-eb2c200ef7d9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:55.300541 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3790a43-0d1e-4eca-8944-01de2ce16611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:55.301566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3790a43-0d1e-4eca-8944-01de2ce16611 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:55.302070 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3790a43-0d1e-4eca-8944-01de2ce16611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:55.309577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3790a43-0d1e-4eca-8944-01de2ce16611 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:55.309577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3790a43-0d1e-4eca-8944-01de2ce16611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:55.310079 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 833/1677] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:13:55 2022] POST /identity/v3/users => generated 290 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:55.311464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7718e-3226-4456-b7d5-384d49b57c06 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:55.311735 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3790a43-0d1e-4eca-8944-01de2ce16611 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:55.314005 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7718e-3226-4456-b7d5-384d49b57c06 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:55.314289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be7718e-3226-4456-b7d5-384d49b57c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:55.358432 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 845/1678] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:13:54 2022] POST /identity/v3/users => generated 303 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:13:55.359615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3790a43-0d1e-4eca-8944-01de2ce16611 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:55.378200 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7be7718e-3226-4456-b7d5-384d49b57c06 None None] MFA Rules not processed for user `fed44db7dc014719b5ed0399c715435b`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:55.380274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7be7718e-3226-4456-b7d5-384d49b57c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:55.380938 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3790a43-0d1e-4eca-8944-01de2ce16611 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:55.391779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce982668-c537-443e-9acd-ef879189315f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:55.392498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce982668-c537-443e-9acd-ef879189315f admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:55.392783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce982668-c537-443e-9acd-ef879189315f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:55.393089 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce982668-c537-443e-9acd-ef879189315f admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:55.393845 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce982668-c537-443e-9acd-ef879189315f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:55.397192 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce982668-c537-443e-9acd-ef879189315f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:55.439347 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 846/1679] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:55.441624 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce982668-c537-443e-9acd-ef879189315f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:55.466636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce982668-c537-443e-9acd-ef879189315f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:55.472487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861434e-5d95-4abc-a610-f19579216440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:55.477878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861434e-5d95-4abc-a610-f19579216440 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:55.478394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861434e-5d95-4abc-a610-f19579216440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:55.478838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861434e-5d95-4abc-a610-f19579216440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:55.479748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861434e-5d95-4abc-a610-f19579216440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:55.481547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861434e-5d95-4abc-a610-f19579216440 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:55.487591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:55.529825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 847/1680] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:55.530792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861434e-5d95-4abc-a610-f19579216440 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:55.542970 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 834/1681] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:55.545080 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7718e-3226-4456-b7d5-384d49b57c06 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:55.547384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861434e-5d95-4abc-a610-f19579216440 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:55.575088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17c64c-0d12-4676-aebb-9989a2cd2cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:55.576020 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17c64c-0d12-4676-aebb-9989a2cd2cda admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:55.576308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17c64c-0d12-4676-aebb-9989a2cd2cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:55.576586 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17c64c-0d12-4676-aebb-9989a2cd2cda admin admin] PATH_INFO: `/v3/projects/9c3a985fa4bc49d89eb36e39eaa88b69/users/5ef09e3f212c4959a7155ee3f129bf55/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:55.580374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be7718e-3226-4456-b7d5-384d49b57c06 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:55.598293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97b0c33-f406-477f-a897-750118b4b8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:55.599817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b0c33-f406-477f-a897-750118b4b8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:55.599817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b0c33-f406-477f-a897-750118b4b8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:55.599817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97b0c33-f406-477f-a897-750118b4b8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:55.600298 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b0c33-f406-477f-a897-750118b4b8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:55.602937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97b0c33-f406-477f-a897-750118b4b8e7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:55.611403 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e17c64c-0d12-4676-aebb-9989a2cd2cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c3a985fa4bc49d89eb36e39eaa88b69, user_id=5ef09e3f212c4959a7155ee3f129bf55, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:55.617747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e17c64c-0d12-4676-aebb-9989a2cd2cda admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:55.636956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 835/1682] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:55.638398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28864a-db62-4d62-ab78-5aa7a8b24880 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:55.638684 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28864a-db62-4d62-ab78-5aa7a8b24880 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:55.638957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf28864a-db62-4d62-ab78-5aa7a8b24880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:55.662201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 848/1683] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:55 2022] PUT /identity/v3/projects/9c3a985fa4bc49d89eb36e39eaa88b69/users/5ef09e3f212c4959a7155ee3f129bf55/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:55.663303 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17c64c-0d12-4676-aebb-9989a2cd2cda admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:55.702556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-bf28864a-db62-4d62-ab78-5aa7a8b24880 None None] MFA Rules not processed for user `fed44db7dc014719b5ed0399c715435b`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:55.705175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-bf28864a-db62-4d62-ab78-5aa7a8b24880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:55.830601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:55.852316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 836/1684] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:55.853053 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17c64c-0d12-4676-aebb-9989a2cd2cda admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:55.854314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28864a-db62-4d62-ab78-5aa7a8b24880 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:55.865155 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487aeba-afdc-43fa-ae9c-1199d9a7e28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:55.866237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487aeba-afdc-43fa-ae9c-1199d9a7e28e admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:55.867116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487aeba-afdc-43fa-ae9c-1199d9a7e28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:55.867495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0487aeba-afdc-43fa-ae9c-1199d9a7e28e admin admin] PATH_INFO: `/v3/projects/6e6e705e22c54aeeb45c8889952bcb2a/users/ae42bbb0fb2b4ce0a75bda80e8d6faa2/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:55.881620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf28864a-db62-4d62-ab78-5aa7a8b24880 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:55.885272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487aeba-afdc-43fa-ae9c-1199d9a7e28e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e6e705e22c54aeeb45c8889952bcb2a, user_id=ae42bbb0fb2b4ce0a75bda80e8d6faa2, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:55.888248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0487aeba-afdc-43fa-ae9c-1199d9a7e28e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:55.894037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af51adf-3a65-4c5c-887b-e6e19c818777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:55.894724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af51adf-3a65-4c5c-887b-e6e19c818777 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:55.895004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af51adf-3a65-4c5c-887b-e6e19c818777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:55.895276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af51adf-3a65-4c5c-887b-e6e19c818777 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:55.896019 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af51adf-3a65-4c5c-887b-e6e19c818777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:55.897704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af51adf-3a65-4c5c-887b-e6e19c818777 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:55.917565 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 849/1685] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:55 2022] PUT /identity/v3/projects/6e6e705e22c54aeeb45c8889952bcb2a/users/ae42bbb0fb2b4ce0a75bda80e8d6faa2/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:55.923129 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487aeba-afdc-43fa-ae9c-1199d9a7e28e admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:55.924575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 837/1686] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:13:55.925957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279c99a-8dfa-48d2-999f-486cc26e8f2d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:55.926243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279c99a-8dfa-48d2-999f-486cc26e8f2d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:55.926562 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3279c99a-8dfa-48d2-999f-486cc26e8f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:55.929539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0487aeba-afdc-43fa-ae9c-1199d9a7e28e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:55.962242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3279c99a-8dfa-48d2-999f-486cc26e8f2d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:55.963471 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3279c99a-8dfa-48d2-999f-486cc26e8f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:56.131229 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0487aeba-afdc-43fa-ae9c-1199d9a7e28e admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:56.138500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3282e1e-e212-47d7-9f65-87e8dd102df9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:56.139280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3282e1e-e212-47d7-9f65-87e8dd102df9 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:56.139983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3282e1e-e212-47d7-9f65-87e8dd102df9 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:56.139983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3282e1e-e212-47d7-9f65-87e8dd102df9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:56.160597 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 838/1687] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:56.161493 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279c99a-8dfa-48d2-999f-486cc26e8f2d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:56.172229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3279c99a-8dfa-48d2-999f-486cc26e8f2d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:56.177568 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532691c4-40ec-4cc6-958e-2010032eca4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:56.178774 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532691c4-40ec-4cc6-958e-2010032eca4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:56.179074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532691c4-40ec-4cc6-958e-2010032eca4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:56.179349 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532691c4-40ec-4cc6-958e-2010032eca4e admin admin] PATH_INFO: `/v3/projects/9c3a985fa4bc49d89eb36e39eaa88b69/users/5ef09e3f212c4959a7155ee3f129bf55/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:56.182558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532691c4-40ec-4cc6-958e-2010032eca4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c3a985fa4bc49d89eb36e39eaa88b69, user_id=5ef09e3f212c4959a7155ee3f129bf55, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:56.185273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532691c4-40ec-4cc6-958e-2010032eca4e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:56.220117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 839/1688] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:13:56 2022] PUT /identity/v3/projects/9c3a985fa4bc49d89eb36e39eaa88b69/users/5ef09e3f212c4959a7155ee3f129bf55/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:13:56.222006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf6f57-a6cf-48b9-9108-bcc4c65ff587 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:56.222317 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf6f57-a6cf-48b9-9108-bcc4c65ff587 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:56.222590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf6f57-a6cf-48b9-9108-bcc4c65ff587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:56.274212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-57cf6f57-a6cf-48b9-9108-bcc4c65ff587 None None] MFA Rules not processed for user `5ef09e3f212c4959a7155ee3f129bf55`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:56.276327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-57cf6f57-a6cf-48b9-9108-bcc4c65ff587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:56.283013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:56.309661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3282e1e-e212-47d7-9f65-87e8dd102df9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:56.313945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3282e1e-e212-47d7-9f65-87e8dd102df9 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:56.334431 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 850/1689] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:13:56.398730 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 840/1690] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:56 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:13:56.407067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3282e1e-e212-47d7-9f65-87e8dd102df9 service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:56.408538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f3282e1e-e212-47d7-9f65-87e8dd102df9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:56.470372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf6f57-a6cf-48b9-9108-bcc4c65ff587 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:56.471714 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-57cf6f57-a6cf-48b9-9108-bcc4c65ff587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:56.594203 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3282e1e-e212-47d7-9f65-87e8dd102df9 service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:56.655814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf6f57-a6cf-48b9-9108-bcc4c65ff587 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:56.670290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e585dd-2da8-49d8-8e56-13560096ebb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:56.685200 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e585dd-2da8-49d8-8e56-13560096ebb8 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:56.685799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e585dd-2da8-49d8-8e56-13560096ebb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:56.686201 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e585dd-2da8-49d8-8e56-13560096ebb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:56.697422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fa010-8c4c-459d-b183-fc13c94f2402 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:56.698266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fa010-8c4c-459d-b183-fc13c94f2402 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:56.698917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fa010-8c4c-459d-b183-fc13c94f2402 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:56.699207 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fa010-8c4c-459d-b183-fc13c94f2402 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:56.708614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:56.726537 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fa010-8c4c-459d-b183-fc13c94f2402 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:56.732316 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fa010-8c4c-459d-b183-fc13c94f2402 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:56.765014 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 851/1691] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:56 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:13:56.837652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e585dd-2da8-49d8-8e56-13560096ebb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:56.839557 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e585dd-2da8-49d8-8e56-13560096ebb8 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:56.877853 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 841/1692] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:13:57.772958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e9c21-c2fa-4528-8d90-cb6700d8b7af None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:57.773669 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e9c21-c2fa-4528-8d90-cb6700d8b7af None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:57.773669 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e9c21-c2fa-4528-8d90-cb6700d8b7af None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:57.775342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 852/1693] 10.222.0.61 () {54 vars in 912 bytes} [Thu Jul 14 10:13:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:13:57.784883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d1ec7-3180-45db-808c-eb0ade76645d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:57.786723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d1ec7-3180-45db-808c-eb0ade76645d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:57.786723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d1ec7-3180-45db-808c-eb0ade76645d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:57.909343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4b1d1ec7-3180-45db-808c-eb0ade76645d None None] MFA Rules not processed for user `cd6d19c4406b4b228065b98a9a074b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:57.910574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4b1d1ec7-3180-45db-808c-eb0ade76645d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:57.931775 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 842/1694] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Jul 14 10:13:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:13:57.956531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e9c21-c2fa-4528-8d90-cb6700d8b7af None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:57.958421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d79e9c21-c2fa-4528-8d90-cb6700d8b7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:57.984070 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e9c21-c2fa-4528-8d90-cb6700d8b7af None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:58.007716 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768d0d1-db92-43ef-bec5-0e70f62de6a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:58.013334 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768d0d1-db92-43ef-bec5-0e70f62de6a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:58.013334 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768d0d1-db92-43ef-bec5-0e70f62de6a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:58.013334 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a768d0d1-db92-43ef-bec5-0e70f62de6a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:58.033834 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768d0d1-db92-43ef-bec5-0e70f62de6a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:58.035300 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a768d0d1-db92-43ef-bec5-0e70f62de6a9 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:58.066551 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 853/1695] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:13:58.293682 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddcc77-b92c-451b-aeb6-0bffb0a1f1fd None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:58.294216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddcc77-b92c-451b-aeb6-0bffb0a1f1fd None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:58.295176 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddcc77-b92c-451b-aeb6-0bffb0a1f1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:58.379672 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d2ddcc77-b92c-451b-aeb6-0bffb0a1f1fd None None] MFA Rules not processed for user `bff5f2d1367545f2b4c77fe0d9f095ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:58.381175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d2ddcc77-b92c-451b-aeb6-0bffb0a1f1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:58.576387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 843/1696] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:58.593936 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768d0d1-db92-43ef-bec5-0e70f62de6a9 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:58.599079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a768d0d1-db92-43ef-bec5-0e70f62de6a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:58.734304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a768d0d1-db92-43ef-bec5-0e70f62de6a9 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:58.748060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23635ae5-68b6-4d6b-857d-7579268f16d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:58.750919 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23635ae5-68b6-4d6b-857d-7579268f16d0 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:58.751482 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23635ae5-68b6-4d6b-857d-7579268f16d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:58.753009 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23635ae5-68b6-4d6b-857d-7579268f16d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:58.775239 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78805b70-018e-4496-b2f6-1dc3aa83eaa8 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:58.775528 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78805b70-018e-4496-b2f6-1dc3aa83eaa8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:58.776248 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78805b70-018e-4496-b2f6-1dc3aa83eaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:58.777598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23635ae5-68b6-4d6b-857d-7579268f16d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:58.780078 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23635ae5-68b6-4d6b-857d-7579268f16d0 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:58.817566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 854/1697] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:13:58.849959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-78805b70-018e-4496-b2f6-1dc3aa83eaa8 None None] MFA Rules not processed for user `ca339e26746a4feaaa25b14be59479a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:58.851759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-78805b70-018e-4496-b2f6-1dc3aa83eaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:59.086820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 844/1698] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:59.098913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7314034-470e-4e85-92c4-686f1167ed17 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:59.099327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7314034-470e-4e85-92c4-686f1167ed17 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:59.099703 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7314034-470e-4e85-92c4-686f1167ed17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:59.159097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a7314034-470e-4e85-92c4-686f1167ed17 None None] MFA Rules not processed for user `ca339e26746a4feaaa25b14be59479a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:13:59.161504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a7314034-470e-4e85-92c4-686f1167ed17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:13:59.227224 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 855/1699] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:13:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:13:59.261358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78805b70-018e-4496-b2f6-1dc3aa83eaa8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:13:59.283098 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78805b70-018e-4496-b2f6-1dc3aa83eaa8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:13:59.307575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71833d-0aaf-4091-a1e5-c1396023a71f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:13:59.307575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71833d-0aaf-4091-a1e5-c1396023a71f service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:13:59.307575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71833d-0aaf-4091-a1e5-c1396023a71f service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:13:59.307575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71833d-0aaf-4091-a1e5-c1396023a71f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:13:59.334575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:13:59.339531 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71833d-0aaf-4091-a1e5-c1396023a71f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:13:59.341189 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71833d-0aaf-4091-a1e5-c1396023a71f service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:13:59.373318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 845/1700] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:13:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:00.151081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58f62d-594c-4113-96be-b60ce1a6d6e3 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:00.152670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58f62d-594c-4113-96be-b60ce1a6d6e3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:00.153154 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c58f62d-594c-4113-96be-b60ce1a6d6e3 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:00.154914 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 856/1701] 10.222.0.61 () {54 vars in 960 bytes} [Thu Jul 14 10:14:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:14:00.160737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92eb460-0437-4c53-9097-a9074c568de8 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:00.165554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92eb460-0437-4c53-9097-a9074c568de8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:00.165838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92eb460-0437-4c53-9097-a9074c568de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:00.218014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b92eb460-0437-4c53-9097-a9074c568de8 None None] MFA Rules not processed for user `ff4c42a4f4f549378e5bc1cd92f801c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:00.219455 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b92eb460-0437-4c53-9097-a9074c568de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:00.483697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 846/1702] 10.222.0.61 () {58 vars in 1057 bytes} [Thu Jul 14 10:14:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:00.491215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58f62d-594c-4113-96be-b60ce1a6d6e3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:00.503545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391f218-2ba6-42f2-9c80-17bb3266e11b None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:00.504077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391f218-2ba6-42f2-9c80-17bb3266e11b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:00.504498 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9391f218-2ba6-42f2-9c80-17bb3266e11b None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:00.506552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 847/1703] 10.222.0.61 () {54 vars in 912 bytes} [Thu Jul 14 10:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:14:00.519725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36408cf3-ffbf-4c35-99f0-576776a654b0 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:00.520233 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36408cf3-ffbf-4c35-99f0-576776a654b0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:00.520646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36408cf3-ffbf-4c35-99f0-576776a654b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:00.526035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c58f62d-594c-4113-96be-b60ce1a6d6e3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:00.551861 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f656542-95cb-4a2a-a862-d89a03f93b3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff4c42a4f4f549378e5bc1cd92f801c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:00.552539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f656542-95cb-4a2a-a862-d89a03f93b3e service placement] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:00.552822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f656542-95cb-4a2a-a862-d89a03f93b3e service placement] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:00.554508 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-36408cf3-ffbf-4c35-99f0-576776a654b0 None None] MFA Rules not processed for user `cd6d19c4406b4b228065b98a9a074b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:00.556135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f656542-95cb-4a2a-a862-d89a03f93b3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:00.564285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-36408cf3-ffbf-4c35-99f0-576776a654b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:00.564458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4f656542-95cb-4a2a-a862-d89a03f93b3e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:00.575260 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 848/1704] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Jul 14 10:14:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:14:00.601336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36408cf3-ffbf-4c35-99f0-576776a654b0 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:00.618588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36408cf3-ffbf-4c35-99f0-576776a654b0 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:00.622352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f656542-95cb-4a2a-a862-d89a03f93b3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:00.624132 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f656542-95cb-4a2a-a862-d89a03f93b3e service placement] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:00.627955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aeca1b-79f4-4095-9be8-67080d6f4618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:00.629075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeca1b-79f4-4095-9be8-67080d6f4618 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:00.629511 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeca1b-79f4-4095-9be8-67080d6f4618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:00.629928 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aeca1b-79f4-4095-9be8-67080d6f4618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:00.653246 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aeca1b-79f4-4095-9be8-67080d6f4618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:00.658276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aeca1b-79f4-4095-9be8-67080d6f4618 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:00.679866 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 849/1706] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:00.679999 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 857/1706] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:00.683784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649cc41-54ff-424c-bd88-8747571710c8 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:00.684320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649cc41-54ff-424c-bd88-8747571710c8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:00.684516 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e649cc41-54ff-424c-bd88-8747571710c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:00.762187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e649cc41-54ff-424c-bd88-8747571710c8 None None] MFA Rules not processed for user `ae42bbb0fb2b4ce0a75bda80e8d6faa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:00.763741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e649cc41-54ff-424c-bd88-8747571710c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:00.901572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f656542-95cb-4a2a-a862-d89a03f93b3e service placement] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:00.909202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:00.927585 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f656542-95cb-4a2a-a862-d89a03f93b3e service placement] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:00.941686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4d85d-903b-4f9e-88af-58536daf46f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:00.942591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4d85d-903b-4f9e-88af-58536daf46f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:00.942874 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4d85d-903b-4f9e-88af-58536daf46f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:00.943151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4d85d-903b-4f9e-88af-58536daf46f2 admin admin] PATH_INFO: `/v3/users/3b8a4164ad9f4948bd25d3643ccdc12b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:00.944435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4d85d-903b-4f9e-88af-58536daf46f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b8a4164ad9f4948bd25d3643ccdc12b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:00.947417 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4d85d-903b-4f9e-88af-58536daf46f2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:00.997672 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 850/1707] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:00 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:01.002447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649cc41-54ff-424c-bd88-8747571710c8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:01.046321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e649cc41-54ff-424c-bd88-8747571710c8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:01.086782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b65a95-1ebd-48e0-b93c-5f6492cb5f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:01.088111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b65a95-1ebd-48e0-b93c-5f6492cb5f54 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:01.088111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b65a95-1ebd-48e0-b93c-5f6492cb5f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:01.088111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b65a95-1ebd-48e0-b93c-5f6492cb5f54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:01.088625 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b65a95-1ebd-48e0-b93c-5f6492cb5f54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:01.090773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b65a95-1ebd-48e0-b93c-5f6492cb5f54 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:01.140653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d4e4d85d-903b-4f9e-88af-58536daf46f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b8a4164ad9f4948bd25d3643ccdc12b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:01.143260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d4e4d85d-903b-4f9e-88af-58536daf46f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b8a4164ad9f4948bd25d3643ccdc12b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:01.172649 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 851/1708] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:01 2022] POST /identity/v3/projects => generated 403 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:14:01.176410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b65a95-1ebd-48e0-b93c-5f6492cb5f54 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:01.184550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d3b65a95-1ebd-48e0-b93c-5f6492cb5f54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:01.187233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d4e4d85d-903b-4f9e-88af-58536daf46f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b8a4164ad9f4948bd25d3643ccdc12b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:01.219284 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d4e4d85d-903b-4f9e-88af-58536daf46f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b8a4164ad9f4948bd25d3643ccdc12b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:01.231983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d4e4d85d-903b-4f9e-88af-58536daf46f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b8a4164ad9f4948bd25d3643ccdc12b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:01.246164 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 858/1709] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:00 2022] DELETE /identity/v3/users/3b8a4164ad9f4948bd25d3643ccdc12b => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:14:01.250636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ade0c-8d77-493b-9dbb-63518f89200b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:01.251410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ade0c-8d77-493b-9dbb-63518f89200b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:01.251898 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24ade0c-8d77-493b-9dbb-63518f89200b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:01.279287 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:01.286795 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e24ade0c-8d77-493b-9dbb-63518f89200b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:01.293076 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e24ade0c-8d77-493b-9dbb-63518f89200b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:01.339686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b65a95-1ebd-48e0-b93c-5f6492cb5f54 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:01.444127 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 859/1710] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:14:01.445988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ade0c-8d77-493b-9dbb-63518f89200b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:01.449153 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e24ade0c-8d77-493b-9dbb-63518f89200b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:01.485418 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24ade0c-8d77-493b-9dbb-63518f89200b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:01.501223 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc825-897d-410c-ae56-ade51726feba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:01.502442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc825-897d-410c-ae56-ade51726feba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:01.502982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc825-897d-410c-ae56-ade51726feba admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:01.503375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dc825-897d-410c-ae56-ade51726feba admin admin] PATH_INFO: `/v3/users/70b000c2b8f741eeb8199a98f939acbc` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:01.509110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dc825-897d-410c-ae56-ade51726feba admin admin] RBAC: Authorizing `identity:delete_user(user_id=70b000c2b8f741eeb8199a98f939acbc)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:01.511938 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dc825-897d-410c-ae56-ade51726feba admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:01.562440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85cd66-f556-4b7a-b30c-8868a9bd86c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:01.563473 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85cd66-f556-4b7a-b30c-8868a9bd86c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:01.567790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85cd66-f556-4b7a-b30c-8868a9bd86c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:01.568318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85cd66-f556-4b7a-b30c-8868a9bd86c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:01.569369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85cd66-f556-4b7a-b30c-8868a9bd86c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:01.576160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85cd66-f556-4b7a-b30c-8868a9bd86c4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:01.625682 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 852/1711] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:01 2022] POST /identity/v3/users => generated 308 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:01.626776 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85cd66-f556-4b7a-b30c-8868a9bd86c4 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:01.633659 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ee85cd66-f556-4b7a-b30c-8868a9bd86c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:01.634064 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e14dc825-897d-410c-ae56-ade51726feba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70b000c2b8f741eeb8199a98f939acbc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:01.634793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e14dc825-897d-410c-ae56-ade51726feba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70b000c2b8f741eeb8199a98f939acbc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:01.664731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e14dc825-897d-410c-ae56-ade51726feba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70b000c2b8f741eeb8199a98f939acbc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:01.680701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:01.683987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e14dc825-897d-410c-ae56-ade51726feba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70b000c2b8f741eeb8199a98f939acbc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:01.723521 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e14dc825-897d-410c-ae56-ade51726feba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70b000c2b8f741eeb8199a98f939acbc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:01.741815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 860/1712] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:01 2022] DELETE /identity/v3/users/70b000c2b8f741eeb8199a98f939acbc => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:01.743646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc825-897d-410c-ae56-ade51726feba admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:01.745489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e14dc825-897d-410c-ae56-ade51726feba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:01.807781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85cd66-f556-4b7a-b30c-8868a9bd86c4 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:01.942569 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dc825-897d-410c-ae56-ade51726feba admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:01.960949 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4665f-7d1c-4f1e-ac69-18495fae2cfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:01.961516 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51385b-fbdf-4927-b0c6-fdfe68ede996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:01.961749 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4665f-7d1c-4f1e-ac69-18495fae2cfe service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:01.962199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4665f-7d1c-4f1e-ac69-18495fae2cfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:01.962311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51385b-fbdf-4927-b0c6-fdfe68ede996 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:01.962465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e4665f-7d1c-4f1e-ac69-18495fae2cfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:01.962569 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51385b-fbdf-4927-b0c6-fdfe68ede996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:01.962803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51385b-fbdf-4927-b0c6-fdfe68ede996 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:01.967323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51385b-fbdf-4927-b0c6-fdfe68ede996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:01.968974 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51385b-fbdf-4927-b0c6-fdfe68ede996 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:01.977981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d3e4665f-7d1c-4f1e-ac69-18495fae2cfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:01.998964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 861/1713] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:02.000349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c40fd-48e9-4dc5-b5e2-cc71a467a05c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:02.000815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c40fd-48e9-4dc5-b5e2-cc71a467a05c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:02.001138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958c40fd-48e9-4dc5-b5e2-cc71a467a05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:02.013499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4665f-7d1c-4f1e-ac69-18495fae2cfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:02.018734 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e4665f-7d1c-4f1e-ac69-18495fae2cfe service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:02.020308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-958c40fd-48e9-4dc5-b5e2-cc71a467a05c None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:02.021639 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-958c40fd-48e9-4dc5-b5e2-cc71a467a05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:02.043350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 862/1714] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:02.047554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c40fd-48e9-4dc5-b5e2-cc71a467a05c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:02.052951 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-958c40fd-48e9-4dc5-b5e2-cc71a467a05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:02.059995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 853/1715] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:02.062139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4665f-7d1c-4f1e-ac69-18495fae2cfe service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:02.072789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958c40fd-48e9-4dc5-b5e2-cc71a467a05c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:02.078139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7a04e-5bc4-4467-a1f1-9b76d873b9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:02.079263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7a04e-5bc4-4467-a1f1-9b76d873b9a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:02.079970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7a04e-5bc4-4467-a1f1-9b76d873b9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:02.080403 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7a04e-5bc4-4467-a1f1-9b76d873b9a7 admin admin] PATH_INFO: `/v3/users/4d2739e2821d472da275b8bca6a1bd87` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:02.093441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7a04e-5bc4-4467-a1f1-9b76d873b9a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d2739e2821d472da275b8bca6a1bd87)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:02.096043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e4665f-7d1c-4f1e-ac69-18495fae2cfe service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:02.100229 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7a04e-5bc4-4467-a1f1-9b76d873b9a7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:02.111941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fc9f5-e39b-4e0c-a9a1-9cd05132cf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:02.113290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fc9f5-e39b-4e0c-a9a1-9cd05132cf18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:02.113577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fc9f5-e39b-4e0c-a9a1-9cd05132cf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:02.113854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762fc9f5-e39b-4e0c-a9a1-9cd05132cf18 admin admin] PATH_INFO: `/v3/projects/74bbee80b5564c1c86b56d07526537ae/users/f000c546d0334fb9a0a7d9c5b8a4cb25/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:02.173688 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7da7a04e-5bc4-4467-a1f1-9b76d873b9a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d2739e2821d472da275b8bca6a1bd87'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:02.174444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7da7a04e-5bc4-4467-a1f1-9b76d873b9a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d2739e2821d472da275b8bca6a1bd87'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:02.186644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fc9f5-e39b-4e0c-a9a1-9cd05132cf18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74bbee80b5564c1c86b56d07526537ae, user_id=f000c546d0334fb9a0a7d9c5b8a4cb25, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:02.197269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762fc9f5-e39b-4e0c-a9a1-9cd05132cf18 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:02.200548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7da7a04e-5bc4-4467-a1f1-9b76d873b9a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d2739e2821d472da275b8bca6a1bd87'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:02.239228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7da7a04e-5bc4-4467-a1f1-9b76d873b9a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d2739e2821d472da275b8bca6a1bd87'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:02.256171 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 854/1716] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:02 2022] PUT /identity/v3/projects/74bbee80b5564c1c86b56d07526537ae/users/f000c546d0334fb9a0a7d9c5b8a4cb25/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:02.257545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fc9f5-e39b-4e0c-a9a1-9cd05132cf18 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:02.261043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-762fc9f5-e39b-4e0c-a9a1-9cd05132cf18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:02.261622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7da7a04e-5bc4-4467-a1f1-9b76d873b9a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d2739e2821d472da275b8bca6a1bd87'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:02.291618 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 863/1717] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:02 2022] DELETE /identity/v3/users/4d2739e2821d472da275b8bca6a1bd87 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:02.294047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7a04e-5bc4-4467-a1f1-9b76d873b9a7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:02.301963 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7da7a04e-5bc4-4467-a1f1-9b76d873b9a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:02.305984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:02.464672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:02.519530 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762fc9f5-e39b-4e0c-a9a1-9cd05132cf18 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:02.524991 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7a04e-5bc4-4467-a1f1-9b76d873b9a7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:02.544660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e26b70-e7b1-4950-b36c-b0f82235d38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:02.545643 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e26b70-e7b1-4950-b36c-b0f82235d38f admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:02.546082 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e26b70-e7b1-4950-b36c-b0f82235d38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:02.546504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e26b70-e7b1-4950-b36c-b0f82235d38f admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:02.547391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e26b70-e7b1-4950-b36c-b0f82235d38f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:02.550280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e26b70-e7b1-4950-b36c-b0f82235d38f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:02.584787 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 864/1718] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:02.586425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3dd383-28f2-4a09-ada3-240b28b033f8 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:02.586655 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3dd383-28f2-4a09-ada3-240b28b033f8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:02.586948 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3dd383-28f2-4a09-ada3-240b28b033f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:02.659400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646c558-5b53-4371-87dc-ac8f9899b70f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:02.660079 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646c558-5b53-4371-87dc-ac8f9899b70f service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:02.660360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646c558-5b53-4371-87dc-ac8f9899b70f service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:02.660638 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646c558-5b53-4371-87dc-ac8f9899b70f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:02.662786 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6646c558-5b53-4371-87dc-ac8f9899b70f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:02.670816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2d3dd383-28f2-4a09-ada3-240b28b033f8 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:02.672064 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2d3dd383-28f2-4a09-ada3-240b28b033f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:02.677509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646c558-5b53-4371-87dc-ac8f9899b70f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:02.679892 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646c558-5b53-4371-87dc-ac8f9899b70f service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:02.693624 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 855/1719] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:02.694809 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646c558-5b53-4371-87dc-ac8f9899b70f service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:02.707989 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 865/1720] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:02.711255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646c558-5b53-4371-87dc-ac8f9899b70f service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:02.718910 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cf5d0d-2dd8-4831-9cfd-9785dd9a59e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:02.721095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf5d0d-2dd8-4831-9cfd-9785dd9a59e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:02.721583 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf5d0d-2dd8-4831-9cfd-9785dd9a59e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:02.722161 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cf5d0d-2dd8-4831-9cfd-9785dd9a59e1 admin admin] PATH_INFO: `/v3/projects/74bbee80b5564c1c86b56d07526537ae/users/f000c546d0334fb9a0a7d9c5b8a4cb25/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:02.724952 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf5d0d-2dd8-4831-9cfd-9785dd9a59e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74bbee80b5564c1c86b56d07526537ae, user_id=f000c546d0334fb9a0a7d9c5b8a4cb25, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:02.728882 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cf5d0d-2dd8-4831-9cfd-9785dd9a59e1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:02.753029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 856/1721] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:02 2022] PUT /identity/v3/projects/74bbee80b5564c1c86b56d07526537ae/users/f000c546d0334fb9a0a7d9c5b8a4cb25/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:02.768323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3dd383-28f2-4a09-ada3-240b28b033f8 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:02.798767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3dd383-28f2-4a09-ada3-240b28b033f8 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:02.986832 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9951f7-d93b-48fb-900b-deb514ab23e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:02.987516 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9951f7-d93b-48fb-900b-deb514ab23e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:02.987939 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9951f7-d93b-48fb-900b-deb514ab23e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:02.988227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9951f7-d93b-48fb-900b-deb514ab23e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:03.223644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb29921-f163-4ea5-a2c2-e3dfcffad4ea None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:03.224160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb29921-f163-4ea5-a2c2-e3dfcffad4ea None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:03.224240 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb29921-f163-4ea5-a2c2-e3dfcffad4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:03.271765 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8eb29921-f163-4ea5-a2c2-e3dfcffad4ea None None] MFA Rules not processed for user `ca339e26746a4feaaa25b14be59479a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:03.273076 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8eb29921-f163-4ea5-a2c2-e3dfcffad4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:03.277471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9951f7-d93b-48fb-900b-deb514ab23e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:03.278930 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9951f7-d93b-48fb-900b-deb514ab23e6 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:03.325363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 866/1722] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:03.446075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 857/1723] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:03.581497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9ad4b-9b63-476f-8dc8-66567fd3720c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:03.583674 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9ad4b-9b63-476f-8dc8-66567fd3720c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:03.583674 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9ad4b-9b63-476f-8dc8-66567fd3720c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:03.655918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-abc9ad4b-9b63-476f-8dc8-66567fd3720c None None] MFA Rules not processed for user `01a9eebc1c8540daa31109cfc86d5ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:03.658065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb29921-f163-4ea5-a2c2-e3dfcffad4ea None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:03.659539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8eb29921-f163-4ea5-a2c2-e3dfcffad4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:03.661381 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-abc9ad4b-9b63-476f-8dc8-66567fd3720c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:03.691798 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb29921-f163-4ea5-a2c2-e3dfcffad4ea None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:03.709747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f371e18-a36a-42b6-8d91-8427ab599ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:03.711212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f371e18-a36a-42b6-8d91-8427ab599ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:03.711498 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f371e18-a36a-42b6-8d91-8427ab599ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:03.711772 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f371e18-a36a-42b6-8d91-8427ab599ed5 admin admin] PATH_INFO: `/v3/projects/b22f3f4500c24fe9ac26316c2f6b99ec` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:03.721909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f371e18-a36a-42b6-8d91-8427ab599ed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b22f3f4500c24fe9ac26316c2f6b99ec)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:03.723889 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f371e18-a36a-42b6-8d91-8427ab599ed5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:03.872380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7f371e18-a36a-42b6-8d91-8427ab599ed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b22f3f4500c24fe9ac26316c2f6b99ec', 'request_id': 'req-7f371e18-a36a-42b6-8d91-8427ab599ed5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:03.892120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 867/1724] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:03.893154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7f371e18-a36a-42b6-8d91-8427ab599ed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b22f3f4500c24fe9ac26316c2f6b99ec', 'request_id': 'req-7f371e18-a36a-42b6-8d91-8427ab599ed5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:03.899205 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:03.912342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9ad4b-9b63-476f-8dc8-66567fd3720c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:03.919082 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-abc9ad4b-9b63-476f-8dc8-66567fd3720c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:03.920408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7f371e18-a36a-42b6-8d91-8427ab599ed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b22f3f4500c24fe9ac26316c2f6b99ec', 'request_id': 'req-7f371e18-a36a-42b6-8d91-8427ab599ed5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:03.946458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7f371e18-a36a-42b6-8d91-8427ab599ed5 admin admin] The token cache is being invalidate because project b22f3f4500c24fe9ac26316c2f6b99ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:03.946749 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7f371e18-a36a-42b6-8d91-8427ab599ed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:03.952006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7f371e18-a36a-42b6-8d91-8427ab599ed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:03.957643 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 858/1725] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:03 2022] DELETE /identity/v3/projects/b22f3f4500c24fe9ac26316c2f6b99ec => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:14:03.958564 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f371e18-a36a-42b6-8d91-8427ab599ed5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:03.965839 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7f371e18-a36a-42b6-8d91-8427ab599ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:03.984439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:04.117697 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc9ad4b-9b63-476f-8dc8-66567fd3720c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:04.168905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f371e18-a36a-42b6-8d91-8427ab599ed5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:04.182051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6faf45-0e08-4ff2-8991-a4bc45578e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:04.183242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6faf45-0e08-4ff2-8991-a4bc45578e36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:04.183661 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6faf45-0e08-4ff2-8991-a4bc45578e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:04.184088 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6faf45-0e08-4ff2-8991-a4bc45578e36 admin admin] PATH_INFO: `/v3/users/5ef09e3f212c4959a7155ee3f129bf55` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:04.193943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6faf45-0e08-4ff2-8991-a4bc45578e36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ef09e3f212c4959a7155ee3f129bf55)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:04.196422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6faf45-0e08-4ff2-8991-a4bc45578e36 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:04.279628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5009f85-e4a5-4e07-81e1-8cacae5d96bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:04.280315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5009f85-e4a5-4e07-81e1-8cacae5d96bc service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:04.280600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5009f85-e4a5-4e07-81e1-8cacae5d96bc service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:04.280878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5009f85-e4a5-4e07-81e1-8cacae5d96bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:04.290475 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f5009f85-e4a5-4e07-81e1-8cacae5d96bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:04.331402 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1e6faf45-0e08-4ff2-8991-a4bc45578e36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ef09e3f212c4959a7155ee3f129bf55'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:04.336068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1e6faf45-0e08-4ff2-8991-a4bc45578e36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ef09e3f212c4959a7155ee3f129bf55'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:04.382353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1e6faf45-0e08-4ff2-8991-a4bc45578e36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ef09e3f212c4959a7155ee3f129bf55'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:04.402057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1e6faf45-0e08-4ff2-8991-a4bc45578e36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ef09e3f212c4959a7155ee3f129bf55'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:04.418596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1e6faf45-0e08-4ff2-8991-a4bc45578e36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ef09e3f212c4959a7155ee3f129bf55'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:04.431878 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 859/1726] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:03 2022] DELETE /identity/v3/users/5ef09e3f212c4959a7155ee3f129bf55 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:04.454808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5009f85-e4a5-4e07-81e1-8cacae5d96bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:04.456677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5009f85-e4a5-4e07-81e1-8cacae5d96bc service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:04.470301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 868/1727] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:03 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:04.483490 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6887a59-56c7-4968-b8a3-bcab2272ccc0 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:04.484353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6887a59-56c7-4968-b8a3-bcab2272ccc0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:04.484353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6887a59-56c7-4968-b8a3-bcab2272ccc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:04.606160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b6887a59-56c7-4968-b8a3-bcab2272ccc0 None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:04.607736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b6887a59-56c7-4968-b8a3-bcab2272ccc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:04.669056 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:04.735661 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 860/1728] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Jul 14 10:14:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:04.994729 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5009f85-e4a5-4e07-81e1-8cacae5d96bc service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:04.996267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f5009f85-e4a5-4e07-81e1-8cacae5d96bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:05.098664 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:05.144296 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5009f85-e4a5-4e07-81e1-8cacae5d96bc service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:05.210570 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6887a59-56c7-4968-b8a3-bcab2272ccc0 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:05.212396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b6887a59-56c7-4968-b8a3-bcab2272ccc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:05.309573 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab40e-eb04-4ec1-acbd-4f638a89f502 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:05.310510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab40e-eb04-4ec1-acbd-4f638a89f502 service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:05.310913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab40e-eb04-4ec1-acbd-4f638a89f502 service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:05.311265 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fab40e-eb04-4ec1-acbd-4f638a89f502 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:05.323063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fab40e-eb04-4ec1-acbd-4f638a89f502 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:05.325355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fab40e-eb04-4ec1-acbd-4f638a89f502 service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:05.361819 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6887a59-56c7-4968-b8a3-bcab2272ccc0 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:05.366637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 869/1729] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:14:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:14:05.377986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14aada-c852-45f9-a804-255233e7abfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:05.378901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14aada-c852-45f9-a804-255233e7abfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:05.379179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14aada-c852-45f9-a804-255233e7abfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:05.379452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14aada-c852-45f9-a804-255233e7abfd admin admin] PATH_INFO: `/v3/projects/5ea23db89af74a5e8577533cf5c00bb9` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:05.380756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14aada-c852-45f9-a804-255233e7abfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ea23db89af74a5e8577533cf5c00bb9)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:05.383729 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a14aada-c852-45f9-a804-255233e7abfd admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:05.498032 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-0a14aada-c852-45f9-a804-255233e7abfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ea23db89af74a5e8577533cf5c00bb9', 'request_id': 'req-0a14aada-c852-45f9-a804-255233e7abfd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:05.513723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-0a14aada-c852-45f9-a804-255233e7abfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ea23db89af74a5e8577533cf5c00bb9', 'request_id': 'req-0a14aada-c852-45f9-a804-255233e7abfd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:05.539492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-0a14aada-c852-45f9-a804-255233e7abfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ea23db89af74a5e8577533cf5c00bb9', 'request_id': 'req-0a14aada-c852-45f9-a804-255233e7abfd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:05.558452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-0a14aada-c852-45f9-a804-255233e7abfd admin admin] The token cache is being invalidate because project 5ea23db89af74a5e8577533cf5c00bb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:05.558774 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-0a14aada-c852-45f9-a804-255233e7abfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:05.559520 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-0a14aada-c852-45f9-a804-255233e7abfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:05.564242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 861/1730] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:05 2022] DELETE /identity/v3/projects/5ea23db89af74a5e8577533cf5c00bb9 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:05.565639 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab40e-eb04-4ec1-acbd-4f638a89f502 service swift] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:05.592204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fab40e-eb04-4ec1-acbd-4f638a89f502 service swift] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:05.766393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ec958-e95b-432d-938e-ee237f6d4855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:05.767410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ec958-e95b-432d-938e-ee237f6d4855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:05.767791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ec958-e95b-432d-938e-ee237f6d4855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:05.768077 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ec958-e95b-432d-938e-ee237f6d4855 admin admin] PATH_INFO: `/v3/users/2e3043d3515c41b88b3bc8ecdeb68176` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:05.770418 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ec958-e95b-432d-938e-ee237f6d4855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e3043d3515c41b88b3bc8ecdeb68176)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:05.773106 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90ec958-e95b-432d-938e-ee237f6d4855 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:05.885191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d90ec958-e95b-432d-938e-ee237f6d4855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e3043d3515c41b88b3bc8ecdeb68176'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:05.886237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d90ec958-e95b-432d-938e-ee237f6d4855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e3043d3515c41b88b3bc8ecdeb68176'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:05.926616 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d90ec958-e95b-432d-938e-ee237f6d4855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e3043d3515c41b88b3bc8ecdeb68176'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:05.929552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:05.942795 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d90ec958-e95b-432d-938e-ee237f6d4855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e3043d3515c41b88b3bc8ecdeb68176'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:05.970462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d90ec958-e95b-432d-938e-ee237f6d4855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e3043d3515c41b88b3bc8ecdeb68176'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:05.979032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 870/1731] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:05 2022] DELETE /identity/v3/users/2e3043d3515c41b88b3bc8ecdeb68176 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:06.272041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14aada-c852-45f9-a804-255233e7abfd admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:06.274884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0a14aada-c852-45f9-a804-255233e7abfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:06.298691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a14aada-c852-45f9-a804-255233e7abfd admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:06.447621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f84779-109c-45fe-8e75-ec10ae1667ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:06.449246 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f84779-109c-45fe-8e75-ec10ae1667ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:06.449733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f84779-109c-45fe-8e75-ec10ae1667ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:06.450222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f84779-109c-45fe-8e75-ec10ae1667ea admin admin] PATH_INFO: `/v3/projects/38008d54e5554fb78cc9ced7e6aa2967` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:06.452441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f84779-109c-45fe-8e75-ec10ae1667ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=38008d54e5554fb78cc9ced7e6aa2967)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:06.468376 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f84779-109c-45fe-8e75-ec10ae1667ea admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:06.483191 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:06.590921 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5fae3-5ee4-497f-bad7-ddc1761d733b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:06.591407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5fae3-5ee4-497f-bad7-ddc1761d733b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:06.591511 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e5fae3-5ee4-497f-bad7-ddc1761d733b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:06.594087 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a8f84779-109c-45fe-8e75-ec10ae1667ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38008d54e5554fb78cc9ced7e6aa2967', 'request_id': 'req-a8f84779-109c-45fe-8e75-ec10ae1667ea'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:06.634933 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a8f84779-109c-45fe-8e75-ec10ae1667ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38008d54e5554fb78cc9ced7e6aa2967', 'request_id': 'req-a8f84779-109c-45fe-8e75-ec10ae1667ea'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:06.674127 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d7e5fae3-5ee4-497f-bad7-ddc1761d733b None None] MFA Rules not processed for user `f000c546d0334fb9a0a7d9c5b8a4cb25`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:06.677708 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a8f84779-109c-45fe-8e75-ec10ae1667ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38008d54e5554fb78cc9ced7e6aa2967', 'request_id': 'req-a8f84779-109c-45fe-8e75-ec10ae1667ea'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:06.678060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d7e5fae3-5ee4-497f-bad7-ddc1761d733b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:06.717726 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a8f84779-109c-45fe-8e75-ec10ae1667ea admin admin] The token cache is being invalidate because project 38008d54e5554fb78cc9ced7e6aa2967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:06.720291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a8f84779-109c-45fe-8e75-ec10ae1667ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:06.721299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a8f84779-109c-45fe-8e75-ec10ae1667ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:06.729346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 862/1732] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:06 2022] DELETE /identity/v3/projects/38008d54e5554fb78cc9ced7e6aa2967 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:06.734605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f84779-109c-45fe-8e75-ec10ae1667ea admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:06.736264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a8f84779-109c-45fe-8e75-ec10ae1667ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:06.915247 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f84779-109c-45fe-8e75-ec10ae1667ea admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:06.922314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ea8d7-a05e-4f84-90b0-b3f7584373a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:06.923230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ea8d7-a05e-4f84-90b0-b3f7584373a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:06.923518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ea8d7-a05e-4f84-90b0-b3f7584373a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:06.923981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ea8d7-a05e-4f84-90b0-b3f7584373a2 admin admin] PATH_INFO: `/v3/users/3c0804f1161c4b66b96bcc406e86c72f` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:06.925337 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ea8d7-a05e-4f84-90b0-b3f7584373a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c0804f1161c4b66b96bcc406e86c72f)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:06.928107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278ea8d7-a05e-4f84-90b0-b3f7584373a2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:06.973799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 871/1733] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:06 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 384 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:14:07.039178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-278ea8d7-a05e-4f84-90b0-b3f7584373a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c0804f1161c4b66b96bcc406e86c72f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:07.040250 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-278ea8d7-a05e-4f84-90b0-b3f7584373a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c0804f1161c4b66b96bcc406e86c72f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:07.058510 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-278ea8d7-a05e-4f84-90b0-b3f7584373a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c0804f1161c4b66b96bcc406e86c72f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:07.071957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-278ea8d7-a05e-4f84-90b0-b3f7584373a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c0804f1161c4b66b96bcc406e86c72f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:07.089720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-278ea8d7-a05e-4f84-90b0-b3f7584373a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c0804f1161c4b66b96bcc406e86c72f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:07.109793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 863/1734] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:06 2022] DELETE /identity/v3/users/3c0804f1161c4b66b96bcc406e86c72f => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:07.739765 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd54032-50b1-4a37-b02a-6aa376a85896 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:07.740279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd54032-50b1-4a37-b02a-6aa376a85896 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:07.740379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd54032-50b1-4a37-b02a-6aa376a85896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:07.789908 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5bd54032-50b1-4a37-b02a-6aa376a85896 None None] MFA Rules not processed for user `bff5f2d1367545f2b4c77fe0d9f095ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:07.805851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5bd54032-50b1-4a37-b02a-6aa376a85896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:07.981107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 872/1735] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:08.990613 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccbee3-7b80-401b-92eb-965d0486ee26 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:08.991876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccbee3-7b80-401b-92eb-965d0486ee26 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:08.992309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccbee3-7b80-401b-92eb-965d0486ee26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:09.042299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-96ccbee3-7b80-401b-92eb-965d0486ee26 None None] MFA Rules not processed for user `cd6d19c4406b4b228065b98a9a074b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:09.044063 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-96ccbee3-7b80-401b-92eb-965d0486ee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:09.062823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:09.221148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 864/1736] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Jul 14 10:14:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:14:09.698954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd54032-50b1-4a37-b02a-6aa376a85896 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:09.701236 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5bd54032-50b1-4a37-b02a-6aa376a85896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:09.723639 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd54032-50b1-4a37-b02a-6aa376a85896 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:09.734642 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a3d22-a802-4a95-9984-f6ae7916eb6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:09.735658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a3d22-a802-4a95-9984-f6ae7916eb6d service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:09.735941 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a3d22-a802-4a95-9984-f6ae7916eb6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:09.736266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a3d22-a802-4a95-9984-f6ae7916eb6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:09.751716 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a3d22-a802-4a95-9984-f6ae7916eb6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:09.752620 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a3d22-a802-4a95-9984-f6ae7916eb6d service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:09.764548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 873/1737] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:10.140490 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccbee3-7b80-401b-92eb-965d0486ee26 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:10.143462 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-96ccbee3-7b80-401b-92eb-965d0486ee26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:10.192227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a3d22-a802-4a95-9984-f6ae7916eb6d service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:10.196367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8e3a3d22-a802-4a95-9984-f6ae7916eb6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:10.562659 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccbee3-7b80-401b-92eb-965d0486ee26 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:10.570579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00bf2b-fee5-4139-bc37-187bdde6cdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:10.571910 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00bf2b-fee5-4139-bc37-187bdde6cdb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:10.572376 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00bf2b-fee5-4139-bc37-187bdde6cdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:10.572904 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00bf2b-fee5-4139-bc37-187bdde6cdb4 admin admin] PATH_INFO: `/v3/projects/9c3a985fa4bc49d89eb36e39eaa88b69` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:10.574839 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00bf2b-fee5-4139-bc37-187bdde6cdb4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c3a985fa4bc49d89eb36e39eaa88b69)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:10.577589 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e00bf2b-fee5-4139-bc37-187bdde6cdb4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:10.584590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a3d22-a802-4a95-9984-f6ae7916eb6d service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:10.600324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17327530-345d-4216-8c0c-4a09c451c3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:10.601294 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17327530-345d-4216-8c0c-4a09c451c3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:10.601575 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17327530-345d-4216-8c0c-4a09c451c3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:10.601846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17327530-345d-4216-8c0c-4a09c451c3fe admin admin] PATH_INFO: `/v3/projects/642f2f01d3814a40be2b7bf16a584486` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:10.603160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17327530-345d-4216-8c0c-4a09c451c3fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=642f2f01d3814a40be2b7bf16a584486)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:10.605324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17327530-345d-4216-8c0c-4a09c451c3fe admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:10.619138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:10.714344 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-17327530-345d-4216-8c0c-4a09c451c3fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '642f2f01d3814a40be2b7bf16a584486', 'request_id': 'req-17327530-345d-4216-8c0c-4a09c451c3fe'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:10.728371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4e00bf2b-fee5-4139-bc37-187bdde6cdb4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c3a985fa4bc49d89eb36e39eaa88b69', 'request_id': 'req-4e00bf2b-fee5-4139-bc37-187bdde6cdb4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:10.735216 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-17327530-345d-4216-8c0c-4a09c451c3fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '642f2f01d3814a40be2b7bf16a584486', 'request_id': 'req-17327530-345d-4216-8c0c-4a09c451c3fe'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:10.743131 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4e00bf2b-fee5-4139-bc37-187bdde6cdb4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c3a985fa4bc49d89eb36e39eaa88b69', 'request_id': 'req-4e00bf2b-fee5-4139-bc37-187bdde6cdb4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:10.751698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-17327530-345d-4216-8c0c-4a09c451c3fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '642f2f01d3814a40be2b7bf16a584486', 'request_id': 'req-17327530-345d-4216-8c0c-4a09c451c3fe'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:10.764510 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4e00bf2b-fee5-4139-bc37-187bdde6cdb4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c3a985fa4bc49d89eb36e39eaa88b69', 'request_id': 'req-4e00bf2b-fee5-4139-bc37-187bdde6cdb4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:10.777946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-17327530-345d-4216-8c0c-4a09c451c3fe admin admin] The token cache is being invalidate because project 642f2f01d3814a40be2b7bf16a584486 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:10.778332 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-17327530-345d-4216-8c0c-4a09c451c3fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:10.779229 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-17327530-345d-4216-8c0c-4a09c451c3fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:10.780970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 874/1738] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:10 2022] DELETE /identity/v3/projects/642f2f01d3814a40be2b7bf16a584486 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:10.782778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4e00bf2b-fee5-4139-bc37-187bdde6cdb4 admin admin] The token cache is being invalidate because project 9c3a985fa4bc49d89eb36e39eaa88b69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:10.783339 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4e00bf2b-fee5-4139-bc37-187bdde6cdb4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:10.784198 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17327530-345d-4216-8c0c-4a09c451c3fe admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:10.784480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4e00bf2b-fee5-4139-bc37-187bdde6cdb4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:10.787288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-17327530-345d-4216-8c0c-4a09c451c3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:10.791845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 865/1739] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:10 2022] DELETE /identity/v3/projects/9c3a985fa4bc49d89eb36e39eaa88b69 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:10.800305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86d633-af21-4fd6-9745-447d05c77317 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:10.800916 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86d633-af21-4fd6-9745-447d05c77317 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:10.801533 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc86d633-af21-4fd6-9745-447d05c77317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:10.842342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-fc86d633-af21-4fd6-9745-447d05c77317 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:10.844615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fc86d633-af21-4fd6-9745-447d05c77317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:10.986565 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:11.025644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 866/1740] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:11.034851 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8492d-c021-4aca-884d-f1ed949956e9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:11.034851 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8492d-c021-4aca-884d-f1ed949956e9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:11.034851 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8492d-c021-4aca-884d-f1ed949956e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:11.073070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-71b8492d-c021-4aca-884d-f1ed949956e9 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:11.074372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-71b8492d-c021-4aca-884d-f1ed949956e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:11.106963 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 867/1741] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:11.108508 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6620a15-4153-47cc-b43b-6d80dd9d4df9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:11.108762 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6620a15-4153-47cc-b43b-6d80dd9d4df9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:11.109080 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6620a15-4153-47cc-b43b-6d80dd9d4df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:11.127781 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17327530-345d-4216-8c0c-4a09c451c3fe admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:11.156520 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727a9a7-e329-4018-a890-a462ce7ef8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:11.159955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727a9a7-e329-4018-a890-a462ce7ef8a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:11.160248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727a9a7-e329-4018-a890-a462ce7ef8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:11.166158 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6727a9a7-e329-4018-a890-a462ce7ef8a1 admin admin] PATH_INFO: `/v3/projects/3adeb06f835f41dcb90e36b1819fc86e` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:11.174369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c6620a15-4153-47cc-b43b-6d80dd9d4df9 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:11.175444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727a9a7-e329-4018-a890-a462ce7ef8a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3adeb06f835f41dcb90e36b1819fc86e)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:11.175711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c6620a15-4153-47cc-b43b-6d80dd9d4df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:11.186215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6727a9a7-e329-4018-a890-a462ce7ef8a1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:11.204310 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 868/1742] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:11.212133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0614521-6fd2-41e6-a38f-9cc2a14fbc27 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:11.212231 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0614521-6fd2-41e6-a38f-9cc2a14fbc27 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:11.214930 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0614521-6fd2-41e6-a38f-9cc2a14fbc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:11.241818 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f0614521-6fd2-41e6-a38f-9cc2a14fbc27 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:11.243322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f0614521-6fd2-41e6-a38f-9cc2a14fbc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:11.285509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 869/1743] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:11.286868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0614521-6fd2-41e6-a38f-9cc2a14fbc27 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:11.317378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0614521-6fd2-41e6-a38f-9cc2a14fbc27 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:11.324917 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc281bb0-d4e0-4867-99a8-c4ab97b73302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:11.326516 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc281bb0-d4e0-4867-99a8-c4ab97b73302 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:11.327010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc281bb0-d4e0-4867-99a8-c4ab97b73302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:11.327550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc281bb0-d4e0-4867-99a8-c4ab97b73302 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:11.328633 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc281bb0-d4e0-4867-99a8-c4ab97b73302 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:14:11.329195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc281bb0-d4e0-4867-99a8-c4ab97b73302 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:11.333587 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc281bb0-d4e0-4867-99a8-c4ab97b73302 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:11.359847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 870/1744] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:14:11.361297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc281bb0-d4e0-4867-99a8-c4ab97b73302 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:11.378615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc281bb0-d4e0-4867-99a8-c4ab97b73302 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:11.385567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd1364-7f6f-40de-a477-1d24474217f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:11.386891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd1364-7f6f-40de-a477-1d24474217f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:11.387656 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd1364-7f6f-40de-a477-1d24474217f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:11.388893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd1364-7f6f-40de-a477-1d24474217f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:11.390004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd1364-7f6f-40de-a477-1d24474217f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:14:11.390636 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd1364-7f6f-40de-a477-1d24474217f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:11.392582 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd1364-7f6f-40de-a477-1d24474217f8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:11.423629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 871/1745] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:14:11.425561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd1364-7f6f-40de-a477-1d24474217f8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:11.438824 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd1364-7f6f-40de-a477-1d24474217f8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:11.472954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6727a9a7-e329-4018-a890-a462ce7ef8a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3adeb06f835f41dcb90e36b1819fc86e', 'request_id': 'req-6727a9a7-e329-4018-a890-a462ce7ef8a1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:11.493665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6727a9a7-e329-4018-a890-a462ce7ef8a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3adeb06f835f41dcb90e36b1819fc86e', 'request_id': 'req-6727a9a7-e329-4018-a890-a462ce7ef8a1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:11.517353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6727a9a7-e329-4018-a890-a462ce7ef8a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3adeb06f835f41dcb90e36b1819fc86e', 'request_id': 'req-6727a9a7-e329-4018-a890-a462ce7ef8a1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:11.560432 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6727a9a7-e329-4018-a890-a462ce7ef8a1 admin admin] The token cache is being invalidate because project 3adeb06f835f41dcb90e36b1819fc86e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:11.560664 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6727a9a7-e329-4018-a890-a462ce7ef8a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:11.564448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6727a9a7-e329-4018-a890-a462ce7ef8a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:11.566605 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 875/1746] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:10 2022] DELETE /identity/v3/projects/3adeb06f835f41dcb90e36b1819fc86e => generated 0 bytes in 784 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:11.570520 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727a9a7-e329-4018-a890-a462ce7ef8a1 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:11.570907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6727a9a7-e329-4018-a890-a462ce7ef8a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:11.595655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb83a3-5b24-4e7f-8075-f75e79cf72c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:11.597298 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb83a3-5b24-4e7f-8075-f75e79cf72c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:11.597790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb83a3-5b24-4e7f-8075-f75e79cf72c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:11.598260 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb83a3-5b24-4e7f-8075-f75e79cf72c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:11.599254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb83a3-5b24-4e7f-8075-f75e79cf72c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:11.602387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb83a3-5b24-4e7f-8075-f75e79cf72c0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:11.654707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:11.666326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 872/1747] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:11 2022] POST /identity/v3/projects => generated 363 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:11.669707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9551bd-a5c7-4ce0-9c41-08605e1854fe None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:11.670208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9551bd-a5c7-4ce0-9c41-08605e1854fe None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:11.670482 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9551bd-a5c7-4ce0-9c41-08605e1854fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:11.711016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2a9551bd-a5c7-4ce0-9c41-08605e1854fe None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:11.713568 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2a9551bd-a5c7-4ce0-9c41-08605e1854fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:11.789229 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6727a9a7-e329-4018-a890-a462ce7ef8a1 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:11.829633 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 873/1748] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:11.832685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9551bd-a5c7-4ce0-9c41-08605e1854fe None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:11.836433 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2a9551bd-a5c7-4ce0-9c41-08605e1854fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:11.860912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9551bd-a5c7-4ce0-9c41-08605e1854fe None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:11.870376 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230f6dd-3c89-4131-9d67-c07f647711ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:11.871514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230f6dd-3c89-4131-9d67-c07f647711ea admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:11.872199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230f6dd-3c89-4131-9d67-c07f647711ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:11.872832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230f6dd-3c89-4131-9d67-c07f647711ea admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:11.874508 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230f6dd-3c89-4131-9d67-c07f647711ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:11.889270 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230f6dd-3c89-4131-9d67-c07f647711ea admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:11.946268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 874/1749] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:11 2022] POST /identity/v3/users => generated 282 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:11.946958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae2382-9869-4249-a84a-30f6f0a6c324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:11.947655 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae2382-9869-4249-a84a-30f6f0a6c324 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:11.947938 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae2382-9869-4249-a84a-30f6f0a6c324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:11.948223 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae2382-9869-4249-a84a-30f6f0a6c324 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:11.949005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae2382-9869-4249-a84a-30f6f0a6c324 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:11.950631 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6e4f2-cbcb-422c-ba95-a70a853be67e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:11.954049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6e4f2-cbcb-422c-ba95-a70a853be67e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:11.955054 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee6e4f2-cbcb-422c-ba95-a70a853be67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:11.958259 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ae2382-9869-4249-a84a-30f6f0a6c324 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:12.018302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8ee6e4f2-cbcb-422c-ba95-a70a853be67e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:12.025338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8ee6e4f2-cbcb-422c-ba95-a70a853be67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:12.029448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 876/1750] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:11 2022] POST /identity/v3/projects => generated 399 bytes in 462 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:12.030653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae2382-9869-4249-a84a-30f6f0a6c324 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:12.073419 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:12.215294 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae2382-9869-4249-a84a-30f6f0a6c324 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:12.226866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578d45d-da22-41bd-8026-440a62719bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:12.227574 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578d45d-da22-41bd-8026-440a62719bd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:12.227889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578d45d-da22-41bd-8026-440a62719bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:12.228107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8578d45d-da22-41bd-8026-440a62719bd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:12.232592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578d45d-da22-41bd-8026-440a62719bd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:12.234664 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8578d45d-da22-41bd-8026-440a62719bd7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:12.246889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 877/1751] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:12.247756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578d45d-da22-41bd-8026-440a62719bd7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:12.259958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8578d45d-da22-41bd-8026-440a62719bd7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:12.265426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba932a-4dd1-409b-bc6e-db281d27b052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:12.266109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba932a-4dd1-409b-bc6e-db281d27b052 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:12.266390 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba932a-4dd1-409b-bc6e-db281d27b052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:12.266665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba932a-4dd1-409b-bc6e-db281d27b052 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:12.267410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba932a-4dd1-409b-bc6e-db281d27b052 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:12.271165 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba932a-4dd1-409b-bc6e-db281d27b052 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:12.278695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 875/1752] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:12.279811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6e4f2-cbcb-422c-ba95-a70a853be67e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:12.313021 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 878/1753] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:12 2022] POST /identity/v3/users => generated 300 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:12.313021 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba932a-4dd1-409b-bc6e-db281d27b052 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:12.350879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee6e4f2-cbcb-422c-ba95-a70a853be67e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:12.352140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba932a-4dd1-409b-bc6e-db281d27b052 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:12.362834 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05920844-5002-4a4d-a6b2-79405e4528b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:12.363541 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05920844-5002-4a4d-a6b2-79405e4528b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:12.363829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05920844-5002-4a4d-a6b2-79405e4528b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:12.364100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05920844-5002-4a4d-a6b2-79405e4528b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:12.364841 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05920844-5002-4a4d-a6b2-79405e4528b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:14:12.365612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05920844-5002-4a4d-a6b2-79405e4528b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:12.371339 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05920844-5002-4a4d-a6b2-79405e4528b1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:12.392863 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e8d44-c561-4c91-8478-2c75db02d84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:12.396675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e8d44-c561-4c91-8478-2c75db02d84d admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:12.397154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e8d44-c561-4c91-8478-2c75db02d84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:12.398037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e8d44-c561-4c91-8478-2c75db02d84d admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:12.401411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e8d44-c561-4c91-8478-2c75db02d84d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:12.403354 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e8d44-c561-4c91-8478-2c75db02d84d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:12.405180 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 879/1754] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:14:12.407856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05920844-5002-4a4d-a6b2-79405e4528b1 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:12.430891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05920844-5002-4a4d-a6b2-79405e4528b1 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:12.443550 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 876/1755] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:12.444958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfcc97-5945-4a6a-b244-39bb3f0e9c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:12.445112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e8d44-c561-4c91-8478-2c75db02d84d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:12.445556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfcc97-5945-4a6a-b244-39bb3f0e9c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:12.445834 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfcc97-5945-4a6a-b244-39bb3f0e9c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:12.446110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abfcc97-5945-4a6a-b244-39bb3f0e9c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:12.447294 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfcc97-5945-4a6a-b244-39bb3f0e9c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:12.457529 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abfcc97-5945-4a6a-b244-39bb3f0e9c57 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:12.473259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e8d44-c561-4c91-8478-2c75db02d84d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:12.476398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 880/1756] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:12.477793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfcc97-5945-4a6a-b244-39bb3f0e9c57 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:12.495282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172ddb7-0a70-4913-83e1-40fb611e1255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:12.497077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172ddb7-0a70-4913-83e1-40fb611e1255 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:12.497365 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172ddb7-0a70-4913-83e1-40fb611e1255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:12.497647 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1172ddb7-0a70-4913-83e1-40fb611e1255 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:12.500200 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172ddb7-0a70-4913-83e1-40fb611e1255 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:12.500375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abfcc97-5945-4a6a-b244-39bb3f0e9c57 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:12.505795 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1172ddb7-0a70-4913-83e1-40fb611e1255 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:12.514725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e67d08-55a3-40b3-8b89-681f236e3c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:12.515248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e67d08-55a3-40b3-8b89-681f236e3c32 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:12.515630 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e67d08-55a3-40b3-8b89-681f236e3c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:12.516290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e67d08-55a3-40b3-8b89-681f236e3c32 admin admin] PATH_INFO: `/v3/projects/1aae8e36148b4c85a763dda2f24deccc/users/b54f0cb2c79047a289c2ef3aac70ed82/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:12.523283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:12.553078 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e67d08-55a3-40b3-8b89-681f236e3c32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1aae8e36148b4c85a763dda2f24deccc, user_id=b54f0cb2c79047a289c2ef3aac70ed82, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:12.555991 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e67d08-55a3-40b3-8b89-681f236e3c32 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:12.579782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 877/1758] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:12 2022] POST /identity/v3/projects => generated 365 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:12.580115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 881/1758] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:12 2022] PUT /identity/v3/projects/1aae8e36148b4c85a763dda2f24deccc/users/b54f0cb2c79047a289c2ef3aac70ed82/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:12.581981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172ddb7-0a70-4913-83e1-40fb611e1255 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:12.582912 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e67d08-55a3-40b3-8b89-681f236e3c32 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:12.734064 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e67d08-55a3-40b3-8b89-681f236e3c32 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:12.817574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1172ddb7-0a70-4913-83e1-40fb611e1255 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:12.830910 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae5a14-654a-4ea1-bf11-282c6d580544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:12.831714 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae5a14-654a-4ea1-bf11-282c6d580544 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:12.832315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae5a14-654a-4ea1-bf11-282c6d580544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:12.834053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae5a14-654a-4ea1-bf11-282c6d580544 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:12.835558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae5a14-654a-4ea1-bf11-282c6d580544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:12.838154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae5a14-654a-4ea1-bf11-282c6d580544 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:12.878331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 878/1759] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:12.880071 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f9b10-bb85-4290-b8e5-d699c2177bfd None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:12.883899 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f9b10-bb85-4290-b8e5-d699c2177bfd None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:12.884365 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1f9b10-bb85-4290-b8e5-d699c2177bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:12.900150 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bdf024-3d7c-405b-a070-53c8e077b7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:12.900736 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bdf024-3d7c-405b-a070-53c8e077b7b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:12.901052 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bdf024-3d7c-405b-a070-53c8e077b7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:12.901413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bdf024-3d7c-405b-a070-53c8e077b7b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:12.902140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bdf024-3d7c-405b-a070-53c8e077b7b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:12.904501 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bdf024-3d7c-405b-a070-53c8e077b7b1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:12.957272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 882/1760] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:12 2022] POST /identity/v3/users => generated 283 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:12.958209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bdf024-3d7c-405b-a070-53c8e077b7b1 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:12.973947 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bdf024-3d7c-405b-a070-53c8e077b7b1 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:12.980285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0b1f9b10-bb85-4290-b8e5-d699c2177bfd None None] MFA Rules not processed for user `b54f0cb2c79047a289c2ef3aac70ed82`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:12.981942 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0b1f9b10-bb85-4290-b8e5-d699c2177bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:12.986979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4529f99-03f5-490e-a665-12804657d5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:12.988580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4529f99-03f5-490e-a665-12804657d5c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:12.988580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4529f99-03f5-490e-a665-12804657d5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:12.988580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4529f99-03f5-490e-a665-12804657d5c4 admin admin] PATH_INFO: `/v3/projects/4cf960aa799c4f3e9ee822e4c746f86b/users/c962396d5c4e4d0bae8d21eb9763cb18/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:13.065997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4529f99-03f5-490e-a665-12804657d5c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cf960aa799c4f3e9ee822e4c746f86b, user_id=c962396d5c4e4d0bae8d21eb9763cb18, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:13.071646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4529f99-03f5-490e-a665-12804657d5c4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:13.107223 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 883/1761] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:12 2022] PUT /identity/v3/projects/4cf960aa799c4f3e9ee822e4c746f86b/users/c962396d5c4e4d0bae8d21eb9763cb18/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:13.109689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4529f99-03f5-490e-a665-12804657d5c4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:13.233250 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 879/1762] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:12 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:13.253269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3acd5-51d2-4c69-a970-f240775178fe None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:13.254066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3acd5-51d2-4c69-a970-f240775178fe None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:13.255713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f3acd5-51d2-4c69-a970-f240775178fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:13.322557 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:13.331592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4529f99-03f5-490e-a665-12804657d5c4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:13.349358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4762a-c789-4a49-9773-5568224c723c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:13.350053 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4762a-c789-4a49-9773-5568224c723c admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:13.350323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4762a-c789-4a49-9773-5568224c723c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:13.351201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4762a-c789-4a49-9773-5568224c723c admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:13.351866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4762a-c789-4a49-9773-5568224c723c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:13.353500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4762a-c789-4a49-9773-5568224c723c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:13.375973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:13.389793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 884/1763] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:13.392643 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4762a-c789-4a49-9773-5568224c723c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:13.402178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-11f3acd5-51d2-4c69-a970-f240775178fe None None] MFA Rules not processed for user `c962396d5c4e4d0bae8d21eb9763cb18`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:13.405421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-11f3acd5-51d2-4c69-a970-f240775178fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:13.428953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4762a-c789-4a49-9773-5568224c723c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:13.439862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a743078-7497-46b8-a412-701fc887afda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:13.440476 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a743078-7497-46b8-a412-701fc887afda admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:13.440760 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a743078-7497-46b8-a412-701fc887afda admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:13.441138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a743078-7497-46b8-a412-701fc887afda admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:13.441864 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a743078-7497-46b8-a412-701fc887afda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:13.452171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a743078-7497-46b8-a412-701fc887afda admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:13.520619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 885/1764] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:13 2022] POST /identity/v3/projects => generated 365 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:13.520619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a743078-7497-46b8-a412-701fc887afda admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:13.611536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 880/1765] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:13.612525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3acd5-51d2-4c69-a970-f240775178fe None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:13.677033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f3acd5-51d2-4c69-a970-f240775178fe None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:13.683450 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a743078-7497-46b8-a412-701fc887afda admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:13.690621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6347423-2220-4e6b-924a-34224c1ca3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:13.691492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6347423-2220-4e6b-924a-34224c1ca3bf admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:13.693312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6347423-2220-4e6b-924a-34224c1ca3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:13.693763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6347423-2220-4e6b-924a-34224c1ca3bf admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:13.694863 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6347423-2220-4e6b-924a-34224c1ca3bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:13.705517 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6347423-2220-4e6b-924a-34224c1ca3bf admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:13.709418 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1c4d4-297e-495f-839f-372bd6387d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:13.714792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1c4d4-297e-495f-839f-372bd6387d11 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:13.717043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1c4d4-297e-495f-839f-372bd6387d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:13.717175 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1c4d4-297e-495f-839f-372bd6387d11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:13.722057 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1c4d4-297e-495f-839f-372bd6387d11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:13.723779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1c4d4-297e-495f-839f-372bd6387d11 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:13.763746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 881/1766] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:13 2022] POST /identity/v3/users => generated 289 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:13.766475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6347423-2220-4e6b-924a-34224c1ca3bf admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:13.797543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 886/1767] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:13.800374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1c4d4-297e-495f-839f-372bd6387d11 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:13.807076 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6347423-2220-4e6b-924a-34224c1ca3bf admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:13.822276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de1f42-d720-45d3-9991-826528d588a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:13.826295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de1f42-d720-45d3-9991-826528d588a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:13.826792 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de1f42-d720-45d3-9991-826528d588a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:13.827256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de1f42-d720-45d3-9991-826528d588a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:13.830086 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de1f42-d720-45d3-9991-826528d588a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:13.835959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de1f42-d720-45d3-9991-826528d588a4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:13.847788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1c4d4-297e-495f-839f-372bd6387d11 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:13.875311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40166f9-25f5-48b7-aafa-48d001b5466a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:13.875548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40166f9-25f5-48b7-aafa-48d001b5466a admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:13.875843 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40166f9-25f5-48b7-aafa-48d001b5466a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:13.876122 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40166f9-25f5-48b7-aafa-48d001b5466a admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:13.883784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40166f9-25f5-48b7-aafa-48d001b5466a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:13.885660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40166f9-25f5-48b7-aafa-48d001b5466a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:13.927664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 882/1768] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:13 2022] POST /identity/v3/projects => generated 399 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:13.930422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de1f42-d720-45d3-9991-826528d588a4 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:13.933480 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 887/1769] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:13.934516 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40166f9-25f5-48b7-aafa-48d001b5466a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:13.971691 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40166f9-25f5-48b7-aafa-48d001b5466a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:14.122305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de1f42-d720-45d3-9991-826528d588a4 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:14.136454 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a888bc-734a-4acc-bed7-e18988b0fe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:14.138663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a888bc-734a-4acc-bed7-e18988b0fe4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:14.140552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a888bc-734a-4acc-bed7-e18988b0fe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:14.140856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a888bc-734a-4acc-bed7-e18988b0fe4b admin admin] PATH_INFO: `/v3/projects/931ca54dda44423f8d41a6de2dd0cf30/users/406b026b5ad34434b5f66e448b25503d/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:14.171091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4899f1-fdc6-4561-8c20-511eb621263b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:14.172523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4899f1-fdc6-4561-8c20-511eb621263b admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:14.172843 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4899f1-fdc6-4561-8c20-511eb621263b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:14.173230 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4899f1-fdc6-4561-8c20-511eb621263b admin admin] PATH_INFO: `/v3/projects/9a0a0bd50de54fcb92b101837a94386c/users/e9f63aa805f841ec9ce45f33a61e68e2/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:14.197978 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a888bc-734a-4acc-bed7-e18988b0fe4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=931ca54dda44423f8d41a6de2dd0cf30, user_id=406b026b5ad34434b5f66e448b25503d, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:14.202043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4899f1-fdc6-4561-8c20-511eb621263b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a0a0bd50de54fcb92b101837a94386c, user_id=e9f63aa805f841ec9ce45f33a61e68e2, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:14.204825 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a888bc-734a-4acc-bed7-e18988b0fe4b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:14.216464 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4899f1-fdc6-4561-8c20-511eb621263b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:14.264418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 883/1770] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:13 2022] PUT /identity/v3/projects/931ca54dda44423f8d41a6de2dd0cf30/users/406b026b5ad34434b5f66e448b25503d/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:14.269659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 888/1771] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:13 2022] PUT /identity/v3/projects/9a0a0bd50de54fcb92b101837a94386c/users/e9f63aa805f841ec9ce45f33a61e68e2/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:14.273968 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdada3-eae5-4425-aed8-56c3069f71fc None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:14.274257 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdada3-eae5-4425-aed8-56c3069f71fc None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:14.274532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdada3-eae5-4425-aed8-56c3069f71fc None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:14.275944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 889/1772] 10.222.0.61 () {54 vars in 921 bytes} [Thu Jul 14 10:14:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:14:14.280097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a888bc-734a-4acc-bed7-e18988b0fe4b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:14.286097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c9116-5984-4247-996a-e8d1732daea0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:14.286396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c9116-5984-4247-996a-e8d1732daea0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:14.286667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0c9116-5984-4247-996a-e8d1732daea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:14.379672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4b0c9116-5984-4247-996a-e8d1732daea0 None None] MFA Rules not processed for user `01a9eebc1c8540daa31109cfc86d5ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:14.383134 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4b0c9116-5984-4247-996a-e8d1732daea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:14.479712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:14.529619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:14.541031 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a888bc-734a-4acc-bed7-e18988b0fe4b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:14.557842 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f0bd7-07b9-453d-a364-ad3722083e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:14.559909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f0bd7-07b9-453d-a364-ad3722083e0d admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:14.559909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f0bd7-07b9-453d-a364-ad3722083e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:14.561527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8f0bd7-07b9-453d-a364-ad3722083e0d admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:14.561946 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f0bd7-07b9-453d-a364-ad3722083e0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:14.563877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8f0bd7-07b9-453d-a364-ad3722083e0d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:14.595840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 890/1773] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:14.595840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215dc69-df48-43b5-bc84-aaa659f50c71 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:14.595840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215dc69-df48-43b5-bc84-aaa659f50c71 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:14.595840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215dc69-df48-43b5-bc84-aaa659f50c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:14.613535 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 884/1774] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:14 2022] POST /identity/v3/users => generated 306 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:14.615691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f0bd7-07b9-453d-a364-ad3722083e0d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:14.654032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5215dc69-df48-43b5-bc84-aaa659f50c71 None None] MFA Rules not processed for user `406b026b5ad34434b5f66e448b25503d`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:14.655257 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5215dc69-df48-43b5-bc84-aaa659f50c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:14.662467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8f0bd7-07b9-453d-a364-ad3722083e0d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:14.680534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d960ef84-9a3d-4d45-a3ec-048589051b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:14.682780 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960ef84-9a3d-4d45-a3ec-048589051b2d admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:14.683314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960ef84-9a3d-4d45-a3ec-048589051b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:14.683743 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960ef84-9a3d-4d45-a3ec-048589051b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:14.685862 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960ef84-9a3d-4d45-a3ec-048589051b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:14.690142 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d960ef84-9a3d-4d45-a3ec-048589051b2d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:14.735480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 885/1775] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:14.737034 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b58d8-a868-46f6-99b0-0e1b23e6a278 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:14.737331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b58d8-a868-46f6-99b0-0e1b23e6a278 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:14.737610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164b58d8-a868-46f6-99b0-0e1b23e6a278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:14.755022 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 891/1776] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:14.759822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215dc69-df48-43b5-bc84-aaa659f50c71 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:14.796809 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215dc69-df48-43b5-bc84-aaa659f50c71 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:14.798476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-164b58d8-a868-46f6-99b0-0e1b23e6a278 None None] MFA Rules not processed for user `ff4c42a4f4f549378e5bc1cd92f801c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:14.799679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-164b58d8-a868-46f6-99b0-0e1b23e6a278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:14.812850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e108aac-e2d4-4da6-b539-503a4615d0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:14.813734 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e108aac-e2d4-4da6-b539-503a4615d0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:14.814024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e108aac-e2d4-4da6-b539-503a4615d0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:14.814311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e108aac-e2d4-4da6-b539-503a4615d0b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:14.815198 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e108aac-e2d4-4da6-b539-503a4615d0b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:14.816818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e108aac-e2d4-4da6-b539-503a4615d0b6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:14.843328 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 892/1777] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:14.843328 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e108aac-e2d4-4da6-b539-503a4615d0b6 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:14.876213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e108aac-e2d4-4da6-b539-503a4615d0b6 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:14.888255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df18a4-2d46-40e5-a5b8-d33944eadd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:14.893476 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df18a4-2d46-40e5-a5b8-d33944eadd23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:14.893762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df18a4-2d46-40e5-a5b8-d33944eadd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:14.894042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df18a4-2d46-40e5-a5b8-d33944eadd23 admin admin] PATH_INFO: `/v3/projects/9a0a0bd50de54fcb92b101837a94386c/users/e9f63aa805f841ec9ce45f33a61e68e2/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:14.910133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df18a4-2d46-40e5-a5b8-d33944eadd23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a0a0bd50de54fcb92b101837a94386c, user_id=e9f63aa805f841ec9ce45f33a61e68e2, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:14.920344 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4df18a4-2d46-40e5-a5b8-d33944eadd23 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:14.967606 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 886/1779] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Jul 14 10:14:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:14.969448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 893/1779] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:14 2022] PUT /identity/v3/projects/9a0a0bd50de54fcb92b101837a94386c/users/e9f63aa805f841ec9ce45f33a61e68e2/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:14.970558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df18a4-2d46-40e5-a5b8-d33944eadd23 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:14.970930 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b58d8-a868-46f6-99b0-0e1b23e6a278 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:15.000798 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164b58d8-a868-46f6-99b0-0e1b23e6a278 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:15.166561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035b047-9bec-4d8e-b0d0-d1c282422340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:15.167317 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035b047-9bec-4d8e-b0d0-d1c282422340 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:15.167608 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035b047-9bec-4d8e-b0d0-d1c282422340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:15.167969 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035b047-9bec-4d8e-b0d0-d1c282422340 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:15.169829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035b047-9bec-4d8e-b0d0-d1c282422340 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:15.175322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035b047-9bec-4d8e-b0d0-d1c282422340 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:15.195833 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4df18a4-2d46-40e5-a5b8-d33944eadd23 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:15.205572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb54928-fe4c-498e-8f7c-460190287745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:15.214327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb54928-fe4c-498e-8f7c-460190287745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:15.214327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb54928-fe4c-498e-8f7c-460190287745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:15.214327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb54928-fe4c-498e-8f7c-460190287745 admin admin] PATH_INFO: `/v3/projects/b4c2912dfbb5457ca2a784d60f306866/users/92e9ab62d9544a3fa94df1e046a6e78f/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:15.287590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 887/1780] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:14 2022] POST /identity/v3/projects => generated 363 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:15.296064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70251cbf-9f15-4260-b380-09071c8c2299 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:15.298366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70251cbf-9f15-4260-b380-09071c8c2299 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:15.298660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70251cbf-9f15-4260-b380-09071c8c2299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:15.313780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb54928-fe4c-498e-8f7c-460190287745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c2912dfbb5457ca2a784d60f306866, user_id=92e9ab62d9544a3fa94df1e046a6e78f, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:15.321240 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb54928-fe4c-498e-8f7c-460190287745 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:15.359775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 894/1781] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:14 2022] PUT /identity/v3/projects/b4c2912dfbb5457ca2a784d60f306866/users/92e9ab62d9544a3fa94df1e046a6e78f/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:15.365382 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb54928-fe4c-498e-8f7c-460190287745 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:15.375007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5bb54928-fe4c-498e-8f7c-460190287745 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:15.402008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-70251cbf-9f15-4260-b380-09071c8c2299 None None] MFA Rules not processed for user `e9f63aa805f841ec9ce45f33a61e68e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:15.410466 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-70251cbf-9f15-4260-b380-09071c8c2299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:15.505288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:15.511147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:15.601291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb54928-fe4c-498e-8f7c-460190287745 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:15.610301 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 888/1782] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:15.612289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70251cbf-9f15-4260-b380-09071c8c2299 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:15.613402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9e84c-9c02-44a6-bd94-2307c4bb85c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff4c42a4f4f549378e5bc1cd92f801c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:15.614397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9e84c-9c02-44a6-bd94-2307c4bb85c6 service placement] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:15.616119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9e84c-9c02-44a6-bd94-2307c4bb85c6 service placement] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:15.616636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa9e84c-9c02-44a6-bd94-2307c4bb85c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:15.640004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9e84c-9c02-44a6-bd94-2307c4bb85c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:15.641982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa9e84c-9c02-44a6-bd94-2307c4bb85c6 service placement] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:15.690292 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 895/1783] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:15.691636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9e84c-9c02-44a6-bd94-2307c4bb85c6 service placement] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:15.814869 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70251cbf-9f15-4260-b380-09071c8c2299 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:15.832806 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817bea4-d68e-4652-94b7-fadb2d44cc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:15.833968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817bea4-d68e-4652-94b7-fadb2d44cc11 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:15.834472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817bea4-d68e-4652-94b7-fadb2d44cc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:15.835019 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e817bea4-d68e-4652-94b7-fadb2d44cc11 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:15.836033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817bea4-d68e-4652-94b7-fadb2d44cc11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:15.845171 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e817bea4-d68e-4652-94b7-fadb2d44cc11 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:15.893974 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa9e84c-9c02-44a6-bd94-2307c4bb85c6 service placement] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:15.906083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 889/1784] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:15 2022] POST /identity/v3/users => generated 288 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:15.907105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817bea4-d68e-4652-94b7-fadb2d44cc11 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:15.922149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e817bea4-d68e-4652-94b7-fadb2d44cc11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:15.925719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5cfcf-9129-4c71-a104-932907bcac6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:15.926600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5cfcf-9129-4c71-a104-932907bcac6f admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:15.927048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5cfcf-9129-4c71-a104-932907bcac6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:15.927489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf5cfcf-9129-4c71-a104-932907bcac6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:15.928361 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf5cfcf-9129-4c71-a104-932907bcac6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:15.930439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf5cfcf-9129-4c71-a104-932907bcac6f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:15.975992 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 896/1785] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:15.977648 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5cfcf-9129-4c71-a104-932907bcac6f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:16.016576 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf5cfcf-9129-4c71-a104-932907bcac6f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:16.032745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf78369-0482-49bf-9f4f-7c6328d2d776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:16.033603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf78369-0482-49bf-9f4f-7c6328d2d776 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:16.034009 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf78369-0482-49bf-9f4f-7c6328d2d776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:16.034475 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf78369-0482-49bf-9f4f-7c6328d2d776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:16.035398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf78369-0482-49bf-9f4f-7c6328d2d776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:16.037210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf78369-0482-49bf-9f4f-7c6328d2d776 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:16.077332 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 897/1786] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:16.078458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf78369-0482-49bf-9f4f-7c6328d2d776 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:16.105973 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf78369-0482-49bf-9f4f-7c6328d2d776 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:16.120913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e817bea4-d68e-4652-94b7-fadb2d44cc11 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:16.128777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9218e-fa64-44c3-93dd-14d2d7fa2375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:16.130243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9218e-fa64-44c3-93dd-14d2d7fa2375 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:16.134731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76288a24-6c10-4da7-9377-8aa277d20888 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:16.135262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9218e-fa64-44c3-93dd-14d2d7fa2375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:16.135685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9218e-fa64-44c3-93dd-14d2d7fa2375 admin admin] PATH_INFO: `/v3/projects/b4c2912dfbb5457ca2a784d60f306866/users/92e9ab62d9544a3fa94df1e046a6e78f/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:16.136430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76288a24-6c10-4da7-9377-8aa277d20888 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:16.137873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76288a24-6c10-4da7-9377-8aa277d20888 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:16.138281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76288a24-6c10-4da7-9377-8aa277d20888 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:16.140937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9218e-fa64-44c3-93dd-14d2d7fa2375 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4c2912dfbb5457ca2a784d60f306866, user_id=92e9ab62d9544a3fa94df1e046a6e78f, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:16.153409 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9218e-fa64-44c3-93dd-14d2d7fa2375 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:16.168912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76288a24-6c10-4da7-9377-8aa277d20888 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:16.174642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76288a24-6c10-4da7-9377-8aa277d20888 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:16.195336 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 898/1787] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:16 2022] PUT /identity/v3/projects/b4c2912dfbb5457ca2a784d60f306866/users/92e9ab62d9544a3fa94df1e046a6e78f/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:16.200079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9218e-fa64-44c3-93dd-14d2d7fa2375 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:16.204773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 890/1788] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:16.223893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83363350-bda2-4e13-b09e-80813d39a12b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:16.225759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83363350-bda2-4e13-b09e-80813d39a12b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:16.225759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83363350-bda2-4e13-b09e-80813d39a12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:16.286757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-83363350-bda2-4e13-b09e-80813d39a12b None None] MFA Rules not processed for user `92e9ab62d9544a3fa94df1e046a6e78f`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:16.289446 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-83363350-bda2-4e13-b09e-80813d39a12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:16.379035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9218e-fa64-44c3-93dd-14d2d7fa2375 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:16.403342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd29822-0eac-4245-bc82-de658ca84aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:16.404318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd29822-0eac-4245-bc82-de658ca84aae admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:16.404603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd29822-0eac-4245-bc82-de658ca84aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:16.404876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd29822-0eac-4245-bc82-de658ca84aae admin admin] PATH_INFO: `/v3/projects/bbd80b12d66442b898679bac15e25263/users/813d27c77c4c4f89b0886b660893cbe2/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:16.438893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:16.454217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd29822-0eac-4245-bc82-de658ca84aae admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd80b12d66442b898679bac15e25263, user_id=813d27c77c4c4f89b0886b660893cbe2, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:16.465305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd29822-0eac-4245-bc82-de658ca84aae admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:16.516196 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 899/1789] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:16 2022] PUT /identity/v3/projects/bbd80b12d66442b898679bac15e25263/users/813d27c77c4c4f89b0886b660893cbe2/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:16.521699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd29822-0eac-4245-bc82-de658ca84aae admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:16.521862 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 891/1790] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:16 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:16.534435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83363350-bda2-4e13-b09e-80813d39a12b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:16.604756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:16.736230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83363350-bda2-4e13-b09e-80813d39a12b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:16.742658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd29822-0eac-4245-bc82-de658ca84aae admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:16.753034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373140b-af1d-4dd5-a818-bbb8a0a447c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:16.754140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373140b-af1d-4dd5-a818-bbb8a0a447c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:16.754358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373140b-af1d-4dd5-a818-bbb8a0a447c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:16.754636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373140b-af1d-4dd5-a818-bbb8a0a447c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:16.755526 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373140b-af1d-4dd5-a818-bbb8a0a447c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:16.758294 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373140b-af1d-4dd5-a818-bbb8a0a447c5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:16.777002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 900/1791] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:16.781825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373140b-af1d-4dd5-a818-bbb8a0a447c5 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:16.815559 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373140b-af1d-4dd5-a818-bbb8a0a447c5 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:16.837399 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4386b69-f980-41ed-ac7e-c7f95389c154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:16.838395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4386b69-f980-41ed-ac7e-c7f95389c154 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:16.838696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4386b69-f980-41ed-ac7e-c7f95389c154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:16.839361 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4386b69-f980-41ed-ac7e-c7f95389c154 admin admin] PATH_INFO: `/v3/projects/bbd80b12d66442b898679bac15e25263/users/813d27c77c4c4f89b0886b660893cbe2/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:16.842726 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4386b69-f980-41ed-ac7e-c7f95389c154 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd80b12d66442b898679bac15e25263, user_id=813d27c77c4c4f89b0886b660893cbe2, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:16.847785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4386b69-f980-41ed-ac7e-c7f95389c154 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:16.857393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9c912-ddc7-4f43-b520-45d41e6b0698 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:16.858230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9c912-ddc7-4f43-b520-45d41e6b0698 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:16.860203 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9c912-ddc7-4f43-b520-45d41e6b0698 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:16.860628 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d9c912-ddc7-4f43-b520-45d41e6b0698 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:16.892027 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 901/1792] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:16 2022] PUT /identity/v3/projects/bbd80b12d66442b898679bac15e25263/users/813d27c77c4c4f89b0886b660893cbe2/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:16.892987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c033a50-e4bb-46ba-abc0-3f0f62449be7 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:16.893281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c033a50-e4bb-46ba-abc0-3f0f62449be7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:16.893551 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c033a50-e4bb-46ba-abc0-3f0f62449be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:16.973924 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7c033a50-e4bb-46ba-abc0-3f0f62449be7 None None] MFA Rules not processed for user `813d27c77c4c4f89b0886b660893cbe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:16.975157 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7c033a50-e4bb-46ba-abc0-3f0f62449be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:17.055516 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9c912-ddc7-4f43-b520-45d41e6b0698 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:17.065586 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d9c912-ddc7-4f43-b520-45d41e6b0698 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:17.104710 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 892/1793] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:16 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 574 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:17.211797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 902/1794] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:16 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:14:17.270572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9c912-ddc7-4f43-b520-45d41e6b0698 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:17.278323 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-69d9c912-ddc7-4f43-b520-45d41e6b0698 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:17.320647 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d9c912-ddc7-4f43-b520-45d41e6b0698 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:17.408813 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c033a50-e4bb-46ba-abc0-3f0f62449be7 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:17.609715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863dc9e-5c7b-4b48-bd83-c8389a0a296e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:17.610439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863dc9e-5c7b-4b48-bd83-c8389a0a296e service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:17.610673 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863dc9e-5c7b-4b48-bd83-c8389a0a296e service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:17.610992 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2863dc9e-5c7b-4b48-bd83-c8389a0a296e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:17.653712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:17.674249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c033a50-e4bb-46ba-abc0-3f0f62449be7 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:17.678074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863dc9e-5c7b-4b48-bd83-c8389a0a296e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:17.679675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2863dc9e-5c7b-4b48-bd83-c8389a0a296e service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:17.695616 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07187013-50e7-40b0-a3df-938e0f168782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:17.699183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07187013-50e7-40b0-a3df-938e0f168782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:17.699527 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07187013-50e7-40b0-a3df-938e0f168782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:17.699813 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07187013-50e7-40b0-a3df-938e0f168782 admin admin] PATH_INFO: `/v3/users/b54f0cb2c79047a289c2ef3aac70ed82` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:17.705659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07187013-50e7-40b0-a3df-938e0f168782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b54f0cb2c79047a289c2ef3aac70ed82)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:17.718691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 893/1795] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:17 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:17.720234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863dc9e-5c7b-4b48-bd83-c8389a0a296e service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:17.739281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07187013-50e7-40b0-a3df-938e0f168782 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:17.753029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2863dc9e-5c7b-4b48-bd83-c8389a0a296e service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:17.765886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b770716c-90d8-4b72-8d22-c9ec1c3dfefa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:17.766556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770716c-90d8-4b72-8d22-c9ec1c3dfefa service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:17.766789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770716c-90d8-4b72-8d22-c9ec1c3dfefa service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:17.767064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b770716c-90d8-4b72-8d22-c9ec1c3dfefa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:17.867933 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:17.941558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-07187013-50e7-40b0-a3df-938e0f168782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b54f0cb2c79047a289c2ef3aac70ed82'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:17.943081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-07187013-50e7-40b0-a3df-938e0f168782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b54f0cb2c79047a289c2ef3aac70ed82'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:17.977449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-07187013-50e7-40b0-a3df-938e0f168782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b54f0cb2c79047a289c2ef3aac70ed82'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:18.005356 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770716c-90d8-4b72-8d22-c9ec1c3dfefa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:18.007468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b770716c-90d8-4b72-8d22-c9ec1c3dfefa service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:18.010210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-07187013-50e7-40b0-a3df-938e0f168782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b54f0cb2c79047a289c2ef3aac70ed82'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:18.030383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-07187013-50e7-40b0-a3df-938e0f168782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b54f0cb2c79047a289c2ef3aac70ed82'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:18.039934 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 894/1796] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:17 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:18.042168 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d82ec-68c5-4ed8-bd30-bb6c75e71ba0 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:18.044089 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 903/1797] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:17 2022] DELETE /identity/v3/users/b54f0cb2c79047a289c2ef3aac70ed82 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:18.044784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d82ec-68c5-4ed8-bd30-bb6c75e71ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:18.045704 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07187013-50e7-40b0-a3df-938e0f168782 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:18.047082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d82ec-68c5-4ed8-bd30-bb6c75e71ba0 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:18.066433 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 895/1798] 10.222.0.61 () {54 vars in 921 bytes} [Thu Jul 14 10:14:18 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:14:18.067914 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1f719-8883-453b-a41c-f17c9232dd41 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:18.068203 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1f719-8883-453b-a41c-f17c9232dd41 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:18.070236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af1f719-8883-453b-a41c-f17c9232dd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:18.070906 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-07187013-50e7-40b0-a3df-938e0f168782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:18.127937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6af1f719-8883-453b-a41c-f17c9232dd41 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:18.132652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6af1f719-8883-453b-a41c-f17c9232dd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:18.238106 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07187013-50e7-40b0-a3df-938e0f168782 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:18.248783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ef499-2069-4ff0-be5a-d03e41d8608c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:18.249895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ef499-2069-4ff0-be5a-d03e41d8608c service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:18.250277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ef499-2069-4ff0-be5a-d03e41d8608c service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:18.250628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969ef499-2069-4ff0-be5a-d03e41d8608c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:18.255436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-969ef499-2069-4ff0-be5a-d03e41d8608c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:18.351252 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 896/1799] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:18.352978 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757ed30-47d8-4dc1-b78b-7cbae968a8e0 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:18.355909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757ed30-47d8-4dc1-b78b-7cbae968a8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:18.356414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0757ed30-47d8-4dc1-b78b-7cbae968a8e0 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:18.359391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 897/1800] 10.222.0.61 () {54 vars in 921 bytes} [Thu Jul 14 10:14:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:14:18.457161 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a7fa0-877a-4e36-b767-91bda17fd7b1 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:18.457711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a7fa0-877a-4e36-b767-91bda17fd7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:18.458156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91a7fa0-877a-4e36-b767-91bda17fd7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:18.466335 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ef499-2069-4ff0-be5a-d03e41d8608c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:18.470796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969ef499-2069-4ff0-be5a-d03e41d8608c service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:18.499266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 904/1801] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:18 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:18.501097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ef499-2069-4ff0-be5a-d03e41d8608c service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:18.505065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-969ef499-2069-4ff0-be5a-d03e41d8608c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:18.515580 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a91a7fa0-877a-4e36-b767-91bda17fd7b1 None None] MFA Rules not processed for user `e9f4efa3e2f343c2ac858a277601a7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:18.517601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a91a7fa0-877a-4e36-b767-91bda17fd7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:18.694796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969ef499-2069-4ff0-be5a-d03e41d8608c service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:18.705350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470b316-a85b-4ba3-8322-4e436e76327b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:18.706296 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470b316-a85b-4ba3-8322-4e436e76327b service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:18.706771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470b316-a85b-4ba3-8322-4e436e76327b service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:18.707182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470b316-a85b-4ba3-8322-4e436e76327b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:18.729465 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470b316-a85b-4ba3-8322-4e436e76327b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:18.739324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470b316-a85b-4ba3-8322-4e436e76327b service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:18.768143 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 905/1802] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:18.798085 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 898/1803] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Jul 14 10:14:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:14:18.871397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a373c07-498d-4b62-92b0-cf1a41db9824 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:18.871932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a373c07-498d-4b62-92b0-cf1a41db9824 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:18.872350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a373c07-498d-4b62-92b0-cf1a41db9824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:18.881098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:18.932079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2a373c07-498d-4b62-92b0-cf1a41db9824 None None] MFA Rules not processed for user `e9f4efa3e2f343c2ac858a277601a7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:18.934128 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2a373c07-498d-4b62-92b0-cf1a41db9824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:18.968620 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 906/1804] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Jul 14 10:14:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:19.304427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a7fa0-877a-4e36-b767-91bda17fd7b1 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:19.305979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a91a7fa0-877a-4e36-b767-91bda17fd7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:19.332799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91a7fa0-877a-4e36-b767-91bda17fd7b1 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:19.341843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d538a-e0bc-49ed-8600-2598ce576283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:19.343012 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d538a-e0bc-49ed-8600-2598ce576283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:19.343327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d538a-e0bc-49ed-8600-2598ce576283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:19.343588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d538a-e0bc-49ed-8600-2598ce576283 admin admin] PATH_INFO: `/v3/users/ae42bbb0fb2b4ce0a75bda80e8d6faa2` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:19.349854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d538a-e0bc-49ed-8600-2598ce576283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae42bbb0fb2b4ce0a75bda80e8d6faa2)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:19.352405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d538a-e0bc-49ed-8600-2598ce576283 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:19.408738 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:19.490301 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-095d538a-e0bc-49ed-8600-2598ce576283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae42bbb0fb2b4ce0a75bda80e8d6faa2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:19.491093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-095d538a-e0bc-49ed-8600-2598ce576283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae42bbb0fb2b4ce0a75bda80e8d6faa2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:19.534485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-095d538a-e0bc-49ed-8600-2598ce576283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae42bbb0fb2b4ce0a75bda80e8d6faa2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:19.557763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-095d538a-e0bc-49ed-8600-2598ce576283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae42bbb0fb2b4ce0a75bda80e8d6faa2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:19.569822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-095d538a-e0bc-49ed-8600-2598ce576283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae42bbb0fb2b4ce0a75bda80e8d6faa2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:19.580500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 899/1805] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:19 2022] DELETE /identity/v3/users/ae42bbb0fb2b4ce0a75bda80e8d6faa2 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:19.749533 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a373c07-498d-4b62-92b0-cf1a41db9824 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:19.751042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2a373c07-498d-4b62-92b0-cf1a41db9824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:19.752308 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d538a-e0bc-49ed-8600-2598ce576283 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:19.761099 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-095d538a-e0bc-49ed-8600-2598ce576283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:19.774663 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a373c07-498d-4b62-92b0-cf1a41db9824 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:19.887897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d538a-e0bc-49ed-8600-2598ce576283 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:19.911721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498c89a-b497-4c75-b7ee-1bb814669490 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:19.913026 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498c89a-b497-4c75-b7ee-1bb814669490 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:19.913403 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498c89a-b497-4c75-b7ee-1bb814669490 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:19.914585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4498c89a-b497-4c75-b7ee-1bb814669490 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:19.919416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4498c89a-b497-4c75-b7ee-1bb814669490 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:19.946988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498c89a-b497-4c75-b7ee-1bb814669490 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:19.957352 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4498c89a-b497-4c75-b7ee-1bb814669490 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:19.969906 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c677914-c8a3-4581-b57b-1bf2bb37b6b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:19.970582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c677914-c8a3-4581-b57b-1bf2bb37b6b8 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:19.970858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c677914-c8a3-4581-b57b-1bf2bb37b6b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:19.971128 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c677914-c8a3-4581-b57b-1bf2bb37b6b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:19.978569 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6c677914-c8a3-4581-b57b-1bf2bb37b6b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:19.984422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 900/1806] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:20.035416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c677914-c8a3-4581-b57b-1bf2bb37b6b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:20.035416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c677914-c8a3-4581-b57b-1bf2bb37b6b8 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:20.063854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 907/1807] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:20.934849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498c89a-b497-4c75-b7ee-1bb814669490 service nova] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:20.937820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4498c89a-b497-4c75-b7ee-1bb814669490 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:21.042134 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c677914-c8a3-4581-b57b-1bf2bb37b6b8 service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:21.044170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6c677914-c8a3-4581-b57b-1bf2bb37b6b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:21.088855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4498c89a-b497-4c75-b7ee-1bb814669490 service nova] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:21.117589 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b79594-c191-4077-a586-5f11fe926387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:21.118509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b79594-c191-4077-a586-5f11fe926387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:21.118791 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b79594-c191-4077-a586-5f11fe926387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:21.119068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b79594-c191-4077-a586-5f11fe926387 admin admin] PATH_INFO: `/v3/projects/1aae8e36148b4c85a763dda2f24deccc` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:21.120447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b79594-c191-4077-a586-5f11fe926387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1aae8e36148b4c85a763dda2f24deccc)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:21.122775 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b79594-c191-4077-a586-5f11fe926387 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:21.182827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c677914-c8a3-4581-b57b-1bf2bb37b6b8 service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:21.187160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:21.189118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c3a8c-7904-4476-8202-1ebff82a8851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:21.191091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c3a8c-7904-4476-8202-1ebff82a8851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:21.191582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c3a8c-7904-4476-8202-1ebff82a8851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:21.192522 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4c3a8c-7904-4476-8202-1ebff82a8851 admin admin] PATH_INFO: `/v3/projects/6e6e705e22c54aeeb45c8889952bcb2a` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:21.194439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c3a8c-7904-4476-8202-1ebff82a8851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e6e705e22c54aeeb45c8889952bcb2a)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:21.196800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4c3a8c-7904-4476-8202-1ebff82a8851 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:21.207998 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9b79594-c191-4077-a586-5f11fe926387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1aae8e36148b4c85a763dda2f24deccc', 'request_id': 'req-a9b79594-c191-4077-a586-5f11fe926387'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.217699 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9b79594-c191-4077-a586-5f11fe926387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1aae8e36148b4c85a763dda2f24deccc', 'request_id': 'req-a9b79594-c191-4077-a586-5f11fe926387'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.232405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9b79594-c191-4077-a586-5f11fe926387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1aae8e36148b4c85a763dda2f24deccc', 'request_id': 'req-a9b79594-c191-4077-a586-5f11fe926387'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.247733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9b79594-c191-4077-a586-5f11fe926387 admin admin] The token cache is being invalidate because project 1aae8e36148b4c85a763dda2f24deccc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:21.249049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9b79594-c191-4077-a586-5f11fe926387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.250431 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9b79594-c191-4077-a586-5f11fe926387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.252319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 901/1808] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:20 2022] DELETE /identity/v3/projects/1aae8e36148b4c85a763dda2f24deccc => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:21.257027 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b79594-c191-4077-a586-5f11fe926387 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:21.258973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a9b79594-c191-4077-a586-5f11fe926387 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:21.267701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4b4c3a8c-7904-4476-8202-1ebff82a8851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e6e705e22c54aeeb45c8889952bcb2a', 'request_id': 'req-4b4c3a8c-7904-4476-8202-1ebff82a8851'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.275978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4b4c3a8c-7904-4476-8202-1ebff82a8851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e6e705e22c54aeeb45c8889952bcb2a', 'request_id': 'req-4b4c3a8c-7904-4476-8202-1ebff82a8851'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.284659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4b4c3a8c-7904-4476-8202-1ebff82a8851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e6e705e22c54aeeb45c8889952bcb2a', 'request_id': 'req-4b4c3a8c-7904-4476-8202-1ebff82a8851'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.316001 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4b4c3a8c-7904-4476-8202-1ebff82a8851 admin admin] The token cache is being invalidate because project 6e6e705e22c54aeeb45c8889952bcb2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:21.316598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4b4c3a8c-7904-4476-8202-1ebff82a8851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.317419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4b4c3a8c-7904-4476-8202-1ebff82a8851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.323123 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 908/1809] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:21 2022] DELETE /identity/v3/projects/6e6e705e22c54aeeb45c8889952bcb2a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:14:21.324152 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c3a8c-7904-4476-8202-1ebff82a8851 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:21.325705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4b4c3a8c-7904-4476-8202-1ebff82a8851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:21.354854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:21.359442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b79594-c191-4077-a586-5f11fe926387 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:21.416247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4c3a8c-7904-4476-8202-1ebff82a8851 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:21.421066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2030a94-8cc7-4d3f-bef2-ba3dab50bc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:21.422171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2030a94-8cc7-4d3f-bef2-ba3dab50bc6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:21.422768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2030a94-8cc7-4d3f-bef2-ba3dab50bc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:21.423305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2030a94-8cc7-4d3f-bef2-ba3dab50bc6d admin admin] PATH_INFO: `/v3/users/f000c546d0334fb9a0a7d9c5b8a4cb25` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:21.424656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2030a94-8cc7-4d3f-bef2-ba3dab50bc6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f000c546d0334fb9a0a7d9c5b8a4cb25)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:21.427439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2030a94-8cc7-4d3f-bef2-ba3dab50bc6d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:21.448308 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f93e43-914f-4182-8088-56a9d4460bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:21.449275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f93e43-914f-4182-8088-56a9d4460bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:21.449562 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f93e43-914f-4182-8088-56a9d4460bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:21.449841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f93e43-914f-4182-8088-56a9d4460bdd admin admin] PATH_INFO: `/v3/users/e9f63aa805f841ec9ce45f33a61e68e2` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:21.452053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f93e43-914f-4182-8088-56a9d4460bdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9f63aa805f841ec9ce45f33a61e68e2)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:21.456714 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f93e43-914f-4182-8088-56a9d4460bdd admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:21.490957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c2030a94-8cc7-4d3f-bef2-ba3dab50bc6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f000c546d0334fb9a0a7d9c5b8a4cb25'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.491901 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c2030a94-8cc7-4d3f-bef2-ba3dab50bc6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f000c546d0334fb9a0a7d9c5b8a4cb25'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.507936 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c2030a94-8cc7-4d3f-bef2-ba3dab50bc6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f000c546d0334fb9a0a7d9c5b8a4cb25'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.515333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c2030a94-8cc7-4d3f-bef2-ba3dab50bc6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f000c546d0334fb9a0a7d9c5b8a4cb25'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.516279 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-73f93e43-914f-4182-8088-56a9d4460bdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9f63aa805f841ec9ce45f33a61e68e2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.517078 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-73f93e43-914f-4182-8088-56a9d4460bdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9f63aa805f841ec9ce45f33a61e68e2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.527048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c2030a94-8cc7-4d3f-bef2-ba3dab50bc6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f000c546d0334fb9a0a7d9c5b8a4cb25'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.530886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:21.532415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-73f93e43-914f-4182-8088-56a9d4460bdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9f63aa805f841ec9ce45f33a61e68e2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.541014 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 909/1810] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:21 2022] DELETE /identity/v3/users/f000c546d0334fb9a0a7d9c5b8a4cb25 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:21.550596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-73f93e43-914f-4182-8088-56a9d4460bdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9f63aa805f841ec9ce45f33a61e68e2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.560102 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-73f93e43-914f-4182-8088-56a9d4460bdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9f63aa805f841ec9ce45f33a61e68e2'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:21.567185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 902/1811] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:21 2022] DELETE /identity/v3/users/e9f63aa805f841ec9ce45f33a61e68e2 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:14:21.765303 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2030a94-8cc7-4d3f-bef2-ba3dab50bc6d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:21.766811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c2030a94-8cc7-4d3f-bef2-ba3dab50bc6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:21.841711 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2030a94-8cc7-4d3f-bef2-ba3dab50bc6d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:21.882441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:21.910698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:21.911887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:21.912262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:21.912647 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b admin admin] PATH_INFO: `/v3/projects/74bbee80b5564c1c86b56d07526537ae` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:21.914180 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=74bbee80b5564c1c86b56d07526537ae)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:21.916271 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:21.966006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f93e43-914f-4182-8088-56a9d4460bdd admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:21.969025 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-73f93e43-914f-4182-8088-56a9d4460bdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:21.985022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f93e43-914f-4182-8088-56a9d4460bdd admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:21.992051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02424fb-fa1a-49f4-a1a0-7c8f1bd6ffb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:21.993138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02424fb-fa1a-49f4-a1a0-7c8f1bd6ffb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:21.993427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02424fb-fa1a-49f4-a1a0-7c8f1bd6ffb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:21.993688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02424fb-fa1a-49f4-a1a0-7c8f1bd6ffb9 admin admin] PATH_INFO: `/v3/users/406b026b5ad34434b5f66e448b25503d` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:21.995075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02424fb-fa1a-49f4-a1a0-7c8f1bd6ffb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=406b026b5ad34434b5f66e448b25503d)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:21.997038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02424fb-fa1a-49f4-a1a0-7c8f1bd6ffb9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:22.013440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74bbee80b5564c1c86b56d07526537ae', 'request_id': 'req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:22.020552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74bbee80b5564c1c86b56d07526537ae', 'request_id': 'req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:22.026801 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74bbee80b5564c1c86b56d07526537ae', 'request_id': 'req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:22.046555 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b admin admin] The token cache is being invalidate because project 74bbee80b5564c1c86b56d07526537ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:22.047606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:22.048723 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:22.050849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 910/1812] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:21 2022] DELETE /identity/v3/projects/74bbee80b5564c1c86b56d07526537ae => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:22.052046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:22.056106 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:22.079386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a02424fb-fa1a-49f4-a1a0-7c8f1bd6ffb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '406b026b5ad34434b5f66e448b25503d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:22.080530 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a02424fb-fa1a-49f4-a1a0-7c8f1bd6ffb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '406b026b5ad34434b5f66e448b25503d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:22.095048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a02424fb-fa1a-49f4-a1a0-7c8f1bd6ffb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '406b026b5ad34434b5f66e448b25503d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:22.103442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a02424fb-fa1a-49f4-a1a0-7c8f1bd6ffb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '406b026b5ad34434b5f66e448b25503d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:22.112838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a02424fb-fa1a-49f4-a1a0-7c8f1bd6ffb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '406b026b5ad34434b5f66e448b25503d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:22.121404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 903/1813] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:21 2022] DELETE /identity/v3/users/406b026b5ad34434b5f66e448b25503d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:22.122919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3443167-332a-40f9-b94a-0736378bb7db None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.123271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3443167-332a-40f9-b94a-0736378bb7db None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.123618 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3443167-332a-40f9-b94a-0736378bb7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.128649 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dbb4c4-d30b-4c38-a8e7-8d45ce101f7b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:22.144147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a3443167-332a-40f9-b94a-0736378bb7db None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:22.145465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a3443167-332a-40f9-b94a-0736378bb7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:22.206226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724c02e-2165-4470-ab3e-06b016e777fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:22.207307 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724c02e-2165-4470-ab3e-06b016e777fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.207666 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724c02e-2165-4470-ab3e-06b016e777fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.208142 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724c02e-2165-4470-ab3e-06b016e777fd admin admin] PATH_INFO: `/v3/projects/9a0a0bd50de54fcb92b101837a94386c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.209521 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724c02e-2165-4470-ab3e-06b016e777fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a0a0bd50de54fcb92b101837a94386c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:22.211743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724c02e-2165-4470-ab3e-06b016e777fd admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:22.222008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 904/1814] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:22.223563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13985555-2cce-4c9b-863e-7d8caaf8bde9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.223845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13985555-2cce-4c9b-863e-7d8caaf8bde9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.224125 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13985555-2cce-4c9b-863e-7d8caaf8bde9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.246300 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:22.261187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-13985555-2cce-4c9b-863e-7d8caaf8bde9 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:22.262604 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-13985555-2cce-4c9b-863e-7d8caaf8bde9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:22.272713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 905/1815] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:22.274894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0fb9a-c69b-4a2e-b04d-674f8d717b69 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.277537 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0fb9a-c69b-4a2e-b04d-674f8d717b69 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.278022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a0fb9a-c69b-4a2e-b04d-674f8d717b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.297399 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a724c02e-2165-4470-ab3e-06b016e777fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a0a0bd50de54fcb92b101837a94386c', 'request_id': 'req-a724c02e-2165-4470-ab3e-06b016e777fd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:22.299246 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-89a0fb9a-c69b-4a2e-b04d-674f8d717b69 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:22.300545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-89a0fb9a-c69b-4a2e-b04d-674f8d717b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:22.309489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a724c02e-2165-4470-ab3e-06b016e777fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a0a0bd50de54fcb92b101837a94386c', 'request_id': 'req-a724c02e-2165-4470-ab3e-06b016e777fd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:22.326067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a724c02e-2165-4470-ab3e-06b016e777fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a0a0bd50de54fcb92b101837a94386c', 'request_id': 'req-a724c02e-2165-4470-ab3e-06b016e777fd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:22.346171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a724c02e-2165-4470-ab3e-06b016e777fd admin admin] The token cache is being invalidate because project 9a0a0bd50de54fcb92b101837a94386c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:22.346468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a724c02e-2165-4470-ab3e-06b016e777fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:22.347959 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a724c02e-2165-4470-ab3e-06b016e777fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:22.360720 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 911/1816] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:22 2022] DELETE /identity/v3/projects/9a0a0bd50de54fcb92b101837a94386c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:22.361794 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724c02e-2165-4470-ab3e-06b016e777fd admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:22.367184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a724c02e-2165-4470-ab3e-06b016e777fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:22.370417 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:22.424111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 906/1817] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:22.426348 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5539a5a-3fe1-419c-87cb-dfb23e6d8f93 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.426643 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5539a5a-3fe1-419c-87cb-dfb23e6d8f93 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.426921 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5539a5a-3fe1-419c-87cb-dfb23e6d8f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.487729 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e5539a5a-3fe1-419c-87cb-dfb23e6d8f93 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:22.489559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e5539a5a-3fe1-419c-87cb-dfb23e6d8f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:22.501422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724c02e-2165-4470-ab3e-06b016e777fd admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:22.513342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b210c02-908f-4e38-adbc-f93c8bc3d608 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:22.516113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b210c02-908f-4e38-adbc-f93c8bc3d608 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.516418 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b210c02-908f-4e38-adbc-f93c8bc3d608 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.516725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b210c02-908f-4e38-adbc-f93c8bc3d608 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.517793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 907/1818] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:22.518897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5539a5a-3fe1-419c-87cb-dfb23e6d8f93 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:22.521367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2b210c02-908f-4e38-adbc-f93c8bc3d608 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:22.545666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5539a5a-3fe1-419c-87cb-dfb23e6d8f93 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:22.557452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b210c02-908f-4e38-adbc-f93c8bc3d608 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:22.560021 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b210c02-908f-4e38-adbc-f93c8bc3d608 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:22.560727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a4097-7935-4c2f-8d36-a4a0e7db2a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:22.561818 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a4097-7935-4c2f-8d36-a4a0e7db2a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.562625 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a4097-7935-4c2f-8d36-a4a0e7db2a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.563737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3a4097-7935-4c2f-8d36-a4a0e7db2a03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.564857 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3a4097-7935-4c2f-8d36-a4a0e7db2a03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:14:22.565645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3a4097-7935-4c2f-8d36-a4a0e7db2a03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:22.567842 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3a4097-7935-4c2f-8d36-a4a0e7db2a03 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:22.573538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 912/1819] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:22.575772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2601b1b-52d6-4e45-bbe0-5e1e543a2e75 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.575772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2601b1b-52d6-4e45-bbe0-5e1e543a2e75 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.575967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2601b1b-52d6-4e45-bbe0-5e1e543a2e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.588981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 908/1820] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:14:22.594986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a4097-7935-4c2f-8d36-a4a0e7db2a03 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:22.601139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e2601b1b-52d6-4e45-bbe0-5e1e543a2e75 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:22.602519 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e2601b1b-52d6-4e45-bbe0-5e1e543a2e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:22.610842 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3a4097-7935-4c2f-8d36-a4a0e7db2a03 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:22.616438 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 913/1821] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:22.617075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b94a6-022c-41bf-a822-d2210119aa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:22.617761 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b94a6-022c-41bf-a822-d2210119aa42 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.618049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b94a6-022c-41bf-a822-d2210119aa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.618343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91b94a6-022c-41bf-a822-d2210119aa42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.619061 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b94a6-022c-41bf-a822-d2210119aa42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:22.620752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2601b1b-52d6-4e45-bbe0-5e1e543a2e75 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:22.621354 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91b94a6-022c-41bf-a822-d2210119aa42 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:22.638552 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2601b1b-52d6-4e45-bbe0-5e1e543a2e75 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:22.647940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12152e-6499-446e-bcf3-c944fa70ef9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:22.649698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12152e-6499-446e-bcf3-c944fa70ef9d admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.650750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12152e-6499-446e-bcf3-c944fa70ef9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.652136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12152e-6499-446e-bcf3-c944fa70ef9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.655003 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12152e-6499-446e-bcf3-c944fa70ef9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:14:22.655574 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12152e-6499-446e-bcf3-c944fa70ef9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:22.658149 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12152e-6499-446e-bcf3-c944fa70ef9d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:22.658324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 909/1822] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:22 2022] POST /identity/v3/projects => generated 371 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:22.659175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b94a6-022c-41bf-a822-d2210119aa42 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:22.672690 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 914/1823] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:14:22.677210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12152e-6499-446e-bcf3-c944fa70ef9d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:22.689356 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12152e-6499-446e-bcf3-c944fa70ef9d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:22.732735 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91b94a6-022c-41bf-a822-d2210119aa42 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:22.738811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58addd-5422-471a-a56b-efb74bedc671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:22.739642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58addd-5422-471a-a56b-efb74bedc671 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.740113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58addd-5422-471a-a56b-efb74bedc671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.740454 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58addd-5422-471a-a56b-efb74bedc671 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.742077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58addd-5422-471a-a56b-efb74bedc671 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:22.745194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58addd-5422-471a-a56b-efb74bedc671 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:22.781539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 910/1824] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:22 2022] POST /identity/v3/users => generated 286 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:22.782985 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58addd-5422-471a-a56b-efb74bedc671 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:22.785597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec30a3-3355-4442-9ea7-6a4b5c1a4b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:22.786738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec30a3-3355-4442-9ea7-6a4b5c1a4b42 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.786738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec30a3-3355-4442-9ea7-6a4b5c1a4b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.787015 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec30a3-3355-4442-9ea7-6a4b5c1a4b42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.788226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec30a3-3355-4442-9ea7-6a4b5c1a4b42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:22.790268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec30a3-3355-4442-9ea7-6a4b5c1a4b42 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:22.798839 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58addd-5422-471a-a56b-efb74bedc671 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:22.804804 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3b1a3-79eb-457f-93b8-b54ed5a48ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:22.805971 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3b1a3-79eb-457f-93b8-b54ed5a48ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.806262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3b1a3-79eb-457f-93b8-b54ed5a48ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.806538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d3b1a3-79eb-457f-93b8-b54ed5a48ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.807675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3b1a3-79eb-457f-93b8-b54ed5a48ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:22.809487 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d3b1a3-79eb-457f-93b8-b54ed5a48ffb admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:22.821615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 911/1825] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:22.825667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3b1a3-79eb-457f-93b8-b54ed5a48ffb admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:22.828956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 915/1826] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:22 2022] POST /identity/v3/projects => generated 367 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:14:22.832404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec30a3-3355-4442-9ea7-6a4b5c1a4b42 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:22.837019 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d3b1a3-79eb-457f-93b8-b54ed5a48ffb admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:22.847276 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:22.915248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec30a3-3355-4442-9ea7-6a4b5c1a4b42 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:22.920285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4b039-7f89-4de9-8fb1-0cf2024f1ae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:22.920991 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4b039-7f89-4de9-8fb1-0cf2024f1ae9 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.921276 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4b039-7f89-4de9-8fb1-0cf2024f1ae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.921550 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4b039-7f89-4de9-8fb1-0cf2024f1ae9 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.921655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960ed51-bdc8-4e91-aea6-b7724054521b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:22.922248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4b039-7f89-4de9-8fb1-0cf2024f1ae9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:22.922488 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960ed51-bdc8-4e91-aea6-b7724054521b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.922838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960ed51-bdc8-4e91-aea6-b7724054521b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.923189 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960ed51-bdc8-4e91-aea6-b7724054521b admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.924138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960ed51-bdc8-4e91-aea6-b7724054521b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:22.924328 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4b039-7f89-4de9-8fb1-0cf2024f1ae9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:22.926316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960ed51-bdc8-4e91-aea6-b7724054521b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:22.940160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 912/1827] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:14:22.943447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960ed51-bdc8-4e91-aea6-b7724054521b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:22.953201 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960ed51-bdc8-4e91-aea6-b7724054521b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:22.955817 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 916/1828] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:22 2022] POST /identity/v3/users => generated 284 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:22.959524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4b039-7f89-4de9-8fb1-0cf2024f1ae9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:22.960361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdc1ca4-2c0d-43b5-9e87-7e69d54f6d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:22.961363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc1ca4-2c0d-43b5-9e87-7e69d54f6d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.961713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc1ca4-2c0d-43b5-9e87-7e69d54f6d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.962059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdc1ca4-2c0d-43b5-9e87-7e69d54f6d5d admin admin] PATH_INFO: `/v3/projects/2e6ba76e93d847d78eeacb7888a3fc64/users/ec2462d8ae1544a1a225c5599864fb41/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.973335 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4b039-7f89-4de9-8fb1-0cf2024f1ae9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:22.980386 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac18e39-0c44-441b-8ebd-3ddad7edeaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:22.983201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac18e39-0c44-441b-8ebd-3ddad7edeaea admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:22.983468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac18e39-0c44-441b-8ebd-3ddad7edeaea admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:22.984203 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac18e39-0c44-441b-8ebd-3ddad7edeaea admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:22.984474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac18e39-0c44-441b-8ebd-3ddad7edeaea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:22.989624 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac18e39-0c44-441b-8ebd-3ddad7edeaea admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:22.999904 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc1ca4-2c0d-43b5-9e87-7e69d54f6d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6ba76e93d847d78eeacb7888a3fc64, user_id=ec2462d8ae1544a1a225c5599864fb41, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:23.002523 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdc1ca4-2c0d-43b5-9e87-7e69d54f6d5d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:23.007656 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:23.013627 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 917/1829] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:23.018603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac18e39-0c44-441b-8ebd-3ddad7edeaea admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:23.039600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac18e39-0c44-441b-8ebd-3ddad7edeaea admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:23.051449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 913/1830] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:22 2022] PUT /identity/v3/projects/2e6ba76e93d847d78eeacb7888a3fc64/users/ec2462d8ae1544a1a225c5599864fb41/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:14:23.054168 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402b255-f3fb-477e-b1cc-35dab62694b1 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:23.054447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402b255-f3fb-477e-b1cc-35dab62694b1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:23.054726 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b402b255-f3fb-477e-b1cc-35dab62694b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:23.107982 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b402b255-f3fb-477e-b1cc-35dab62694b1 None None] MFA Rules not processed for user `ec2462d8ae1544a1a225c5599864fb41`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:23.109228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b402b255-f3fb-477e-b1cc-35dab62694b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:23.175935 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104cf02-b5ef-4e74-91c2-ac76419abe94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:23.179330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104cf02-b5ef-4e74-91c2-ac76419abe94 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:23.179887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104cf02-b5ef-4e74-91c2-ac76419abe94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:23.180528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c104cf02-b5ef-4e74-91c2-ac76419abe94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:23.182659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104cf02-b5ef-4e74-91c2-ac76419abe94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:23.185637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c104cf02-b5ef-4e74-91c2-ac76419abe94 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:23.206281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 918/1831] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:23.208590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104cf02-b5ef-4e74-91c2-ac76419abe94 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:23.212316 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c104cf02-b5ef-4e74-91c2-ac76419abe94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:23.221098 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 914/1832] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:23 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:23.222847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402b255-f3fb-477e-b1cc-35dab62694b1 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:23.230409 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c104cf02-b5ef-4e74-91c2-ac76419abe94 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:23.239476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b402b255-f3fb-477e-b1cc-35dab62694b1 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:23.241837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447394e9-9b35-4665-b5d3-e5179bffe36f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:23.242771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447394e9-9b35-4665-b5d3-e5179bffe36f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:23.243055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447394e9-9b35-4665-b5d3-e5179bffe36f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:23.243336 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447394e9-9b35-4665-b5d3-e5179bffe36f admin admin] PATH_INFO: `/v3/projects/931ca54dda44423f8d41a6de2dd0cf30` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:23.249019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447394e9-9b35-4665-b5d3-e5179bffe36f admin admin] RBAC: Authorizing `identity:delete_project(project_id=931ca54dda44423f8d41a6de2dd0cf30)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:23.252104 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447394e9-9b35-4665-b5d3-e5179bffe36f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:23.255299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881cdb0-fcae-4209-a6d7-e79025d37d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:23.257216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881cdb0-fcae-4209-a6d7-e79025d37d0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:23.257511 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881cdb0-fcae-4209-a6d7-e79025d37d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:23.257956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5881cdb0-fcae-4209-a6d7-e79025d37d0b admin admin] PATH_INFO: `/v3/projects/8c062d01b1054cd498c113d2e3894d56/users/57e492f57e304038a184dd6f4dbec3d7/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:23.295938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881cdb0-fcae-4209-a6d7-e79025d37d0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c062d01b1054cd498c113d2e3894d56, user_id=57e492f57e304038a184dd6f4dbec3d7, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:23.300033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5881cdb0-fcae-4209-a6d7-e79025d37d0b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:23.348180 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 915/1833] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:23 2022] PUT /identity/v3/projects/8c062d01b1054cd498c113d2e3894d56/users/57e492f57e304038a184dd6f4dbec3d7/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:23.349120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881cdb0-fcae-4209-a6d7-e79025d37d0b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:23.360371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-447394e9-9b35-4665-b5d3-e5179bffe36f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '931ca54dda44423f8d41a6de2dd0cf30', 'request_id': 'req-447394e9-9b35-4665-b5d3-e5179bffe36f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:23.380599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-447394e9-9b35-4665-b5d3-e5179bffe36f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '931ca54dda44423f8d41a6de2dd0cf30', 'request_id': 'req-447394e9-9b35-4665-b5d3-e5179bffe36f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:23.425446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-447394e9-9b35-4665-b5d3-e5179bffe36f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '931ca54dda44423f8d41a6de2dd0cf30', 'request_id': 'req-447394e9-9b35-4665-b5d3-e5179bffe36f'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:23.460188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-447394e9-9b35-4665-b5d3-e5179bffe36f admin admin] The token cache is being invalidate because project 931ca54dda44423f8d41a6de2dd0cf30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:23.465092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-447394e9-9b35-4665-b5d3-e5179bffe36f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:23.467155 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-447394e9-9b35-4665-b5d3-e5179bffe36f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:23.470761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 919/1834] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:23 2022] DELETE /identity/v3/projects/931ca54dda44423f8d41a6de2dd0cf30 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:23.476220 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0a50e-094c-4f06-a985-9b3718169232 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:23.483730 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0a50e-094c-4f06-a985-9b3718169232 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:23.484070 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0a50e-094c-4f06-a985-9b3718169232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:23.545721 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:23.566867 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-97d0a50e-094c-4f06-a985-9b3718169232 None None] MFA Rules not processed for user `57e492f57e304038a184dd6f4dbec3d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:23.572442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-97d0a50e-094c-4f06-a985-9b3718169232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:23.576488 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5881cdb0-fcae-4209-a6d7-e79025d37d0b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:23.699432 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 920/1835] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:23 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:23.700309 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0a50e-094c-4f06-a985-9b3718169232 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:23.703755 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-97d0a50e-094c-4f06-a985-9b3718169232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:23.731118 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150fdca-6d08-40ae-9ee8-f64e5a2e5d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:23.731802 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150fdca-6d08-40ae-9ee8-f64e5a2e5d14 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:23.732084 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150fdca-6d08-40ae-9ee8-f64e5a2e5d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:23.732420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150fdca-6d08-40ae-9ee8-f64e5a2e5d14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:23.733074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150fdca-6d08-40ae-9ee8-f64e5a2e5d14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:23.742762 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150fdca-6d08-40ae-9ee8-f64e5a2e5d14 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:23.771771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:23.798296 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 916/1836] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:23 2022] POST /identity/v3/projects => generated 373 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:23.798296 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150fdca-6d08-40ae-9ee8-f64e5a2e5d14 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:23.806043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5150fdca-6d08-40ae-9ee8-f64e5a2e5d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:23.852228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0a50e-094c-4f06-a985-9b3718169232 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:23.893246 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150fdca-6d08-40ae-9ee8-f64e5a2e5d14 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:23.899109 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9da18d-3705-44d7-93f4-3591e3bed1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:23.899798 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9da18d-3705-44d7-93f4-3591e3bed1c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:23.900139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9da18d-3705-44d7-93f4-3591e3bed1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:23.901085 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9da18d-3705-44d7-93f4-3591e3bed1c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:23.901085 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9da18d-3705-44d7-93f4-3591e3bed1c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:23.903568 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9da18d-3705-44d7-93f4-3591e3bed1c0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:23.947983 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 917/1837] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:23 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:23.947983 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9da18d-3705-44d7-93f4-3591e3bed1c0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:23.947983 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fc9da18d-3705-44d7-93f4-3591e3bed1c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:23.958847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52182e85-b592-46cb-bd6f-dd35f5bc14bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:23.960055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52182e85-b592-46cb-bd6f-dd35f5bc14bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:23.960513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52182e85-b592-46cb-bd6f-dd35f5bc14bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:23.960835 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52182e85-b592-46cb-bd6f-dd35f5bc14bd admin admin] PATH_INFO: `/v3/users/813d27c77c4c4f89b0886b660893cbe2` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:23.962227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52182e85-b592-46cb-bd6f-dd35f5bc14bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=813d27c77c4c4f89b0886b660893cbe2)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:23.965265 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52182e85-b592-46cb-bd6f-dd35f5bc14bd admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:24.043685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9da18d-3705-44d7-93f4-3591e3bed1c0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:24.046797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-52182e85-b592-46cb-bd6f-dd35f5bc14bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '813d27c77c4c4f89b0886b660893cbe2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:24.047591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-52182e85-b592-46cb-bd6f-dd35f5bc14bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '813d27c77c4c4f89b0886b660893cbe2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:24.050779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:24.055496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de92e9-0805-4cc6-89ec-ac31a193c0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:24.056754 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de92e9-0805-4cc6-89ec-ac31a193c0fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:24.057245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de92e9-0805-4cc6-89ec-ac31a193c0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:24.057675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de92e9-0805-4cc6-89ec-ac31a193c0fb admin admin] PATH_INFO: `/v3/users/c962396d5c4e4d0bae8d21eb9763cb18` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:24.059675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de92e9-0805-4cc6-89ec-ac31a193c0fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c962396d5c4e4d0bae8d21eb9763cb18)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:24.063125 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de92e9-0805-4cc6-89ec-ac31a193c0fb admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:24.076451 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-52182e85-b592-46cb-bd6f-dd35f5bc14bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '813d27c77c4c4f89b0886b660893cbe2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:24.090583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-52182e85-b592-46cb-bd6f-dd35f5bc14bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '813d27c77c4c4f89b0886b660893cbe2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:24.120430 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-52182e85-b592-46cb-bd6f-dd35f5bc14bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '813d27c77c4c4f89b0886b660893cbe2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:24.147116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 921/1838] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:23 2022] DELETE /identity/v3/users/813d27c77c4c4f89b0886b660893cbe2 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:24.148279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52182e85-b592-46cb-bd6f-dd35f5bc14bd admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:24.150789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-52182e85-b592-46cb-bd6f-dd35f5bc14bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:24.166150 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-91de92e9-0805-4cc6-89ec-ac31a193c0fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c962396d5c4e4d0bae8d21eb9763cb18'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:24.166869 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-91de92e9-0805-4cc6-89ec-ac31a193c0fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c962396d5c4e4d0bae8d21eb9763cb18'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:24.191646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-91de92e9-0805-4cc6-89ec-ac31a193c0fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c962396d5c4e4d0bae8d21eb9763cb18'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:24.202082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-91de92e9-0805-4cc6-89ec-ac31a193c0fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c962396d5c4e4d0bae8d21eb9763cb18'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:24.211604 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-91de92e9-0805-4cc6-89ec-ac31a193c0fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c962396d5c4e4d0bae8d21eb9763cb18'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:24.213571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:24.220610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 918/1839] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:23 2022] DELETE /identity/v3/users/c962396d5c4e4d0bae8d21eb9763cb18 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:24.221444 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de92e9-0805-4cc6-89ec-ac31a193c0fb admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:24.225839 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-91de92e9-0805-4cc6-89ec-ac31a193c0fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:24.258749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52182e85-b592-46cb-bd6f-dd35f5bc14bd admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:24.325099 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de92e9-0805-4cc6-89ec-ac31a193c0fb admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:24.332503 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f37506-0aa4-4b72-b20a-733561f50f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:24.334644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f37506-0aa4-4b72-b20a-733561f50f5f admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:24.335669 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f37506-0aa4-4b72-b20a-733561f50f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:24.336767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f37506-0aa4-4b72-b20a-733561f50f5f admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:24.340567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f37506-0aa4-4b72-b20a-733561f50f5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:24.343639 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f37506-0aa4-4b72-b20a-733561f50f5f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:24.383306 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 919/1840] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:24 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:24.387580 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885077e-5bbb-4909-9c4f-3d150fb2d195 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:24.390184 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885077e-5bbb-4909-9c4f-3d150fb2d195 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:24.391500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e885077e-5bbb-4909-9c4f-3d150fb2d195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:24.397102 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919f79c-473c-406d-8550-cc4e89390a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:24.399657 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919f79c-473c-406d-8550-cc4e89390a18 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:24.402579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919f79c-473c-406d-8550-cc4e89390a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:24.403068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919f79c-473c-406d-8550-cc4e89390a18 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:24.406138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919f79c-473c-406d-8550-cc4e89390a18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:24.412346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919f79c-473c-406d-8550-cc4e89390a18 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:24.442640 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e885077e-5bbb-4909-9c4f-3d150fb2d195 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:24.444291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e885077e-5bbb-4909-9c4f-3d150fb2d195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:24.456218 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 922/1841] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:24 2022] POST /identity/v3/users => generated 293 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:24.461928 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919f79c-473c-406d-8550-cc4e89390a18 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:24.468035 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 920/1842] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:24.471308 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885077e-5bbb-4909-9c4f-3d150fb2d195 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:24.479610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e885077e-5bbb-4909-9c4f-3d150fb2d195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:24.493549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919f79c-473c-406d-8550-cc4e89390a18 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:24.518792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23e67d-e971-40bd-8dbf-836a95b98f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:24.520740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23e67d-e971-40bd-8dbf-836a95b98f76 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:24.521751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23e67d-e971-40bd-8dbf-836a95b98f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:24.522273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e885077e-5bbb-4909-9c4f-3d150fb2d195 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:24.522797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23e67d-e971-40bd-8dbf-836a95b98f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:24.524388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23e67d-e971-40bd-8dbf-836a95b98f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:24.527616 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23e67d-e971-40bd-8dbf-836a95b98f76 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:24.534485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baddeaf-1187-44d5-9237-233bb6f6071c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:24.535286 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baddeaf-1187-44d5-9237-233bb6f6071c admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:24.538820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baddeaf-1187-44d5-9237-233bb6f6071c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:24.539233 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baddeaf-1187-44d5-9237-233bb6f6071c admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:24.540113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baddeaf-1187-44d5-9237-233bb6f6071c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:24.542830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baddeaf-1187-44d5-9237-233bb6f6071c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:24.564383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 923/1843] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:24.565517 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23e67d-e971-40bd-8dbf-836a95b98f76 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:24.578904 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 921/1844] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:24.579556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baddeaf-1187-44d5-9237-233bb6f6071c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:24.590489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23e67d-e971-40bd-8dbf-836a95b98f76 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:24.594847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baddeaf-1187-44d5-9237-233bb6f6071c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:24.599829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5488b6-0490-4b4a-b91d-bf14872d6815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:24.601920 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3431d85-d0e9-4106-917e-8a6af6f37d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:24.602322 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5488b6-0490-4b4a-b91d-bf14872d6815 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:24.602840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5488b6-0490-4b4a-b91d-bf14872d6815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:24.603466 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5488b6-0490-4b4a-b91d-bf14872d6815 admin admin] PATH_INFO: `/v3/projects/75a351922ccf4414b873a03ca85ba84b/users/2abfac1ab96e46678da133cf7ffe9cd6/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:24.603690 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3431d85-d0e9-4106-917e-8a6af6f37d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:24.603990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3431d85-d0e9-4106-917e-8a6af6f37d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:24.604416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3431d85-d0e9-4106-917e-8a6af6f37d48 admin admin] PATH_INFO: `/v3/projects/031b08795b8e4375842c78f305700e61/users/23aa82abbe894f37ae99c5070aeec77b/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:24.626998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5488b6-0490-4b4a-b91d-bf14872d6815 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75a351922ccf4414b873a03ca85ba84b, user_id=2abfac1ab96e46678da133cf7ffe9cd6, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:24.627087 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3431d85-d0e9-4106-917e-8a6af6f37d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031b08795b8e4375842c78f305700e61, user_id=23aa82abbe894f37ae99c5070aeec77b, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:24.629603 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3431d85-d0e9-4106-917e-8a6af6f37d48 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:24.632084 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5488b6-0490-4b4a-b91d-bf14872d6815 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:24.659351 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 924/1845] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:24 2022] PUT /identity/v3/projects/75a351922ccf4414b873a03ca85ba84b/users/2abfac1ab96e46678da133cf7ffe9cd6/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:24.660215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5488b6-0490-4b4a-b91d-bf14872d6815 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:24.660825 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 922/1846] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:24 2022] PUT /identity/v3/projects/031b08795b8e4375842c78f305700e61/users/23aa82abbe894f37ae99c5070aeec77b/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:24.662072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2e5488b6-0490-4b4a-b91d-bf14872d6815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:24.662388 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3431d85-d0e9-4106-917e-8a6af6f37d48 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:24.701874 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:24.750354 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3431d85-d0e9-4106-917e-8a6af6f37d48 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:24.756107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5488b6-0490-4b4a-b91d-bf14872d6815 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:24.756641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7377dc6-a228-471d-b6e9-2f3212a7bbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:24.757437 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7377dc6-a228-471d-b6e9-2f3212a7bbf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:24.757781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7377dc6-a228-471d-b6e9-2f3212a7bbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:24.758089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7377dc6-a228-471d-b6e9-2f3212a7bbf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:24.759080 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7377dc6-a228-471d-b6e9-2f3212a7bbf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:24.760741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7377dc6-a228-471d-b6e9-2f3212a7bbf9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:24.772990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 923/1847] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:24.778442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7377dc6-a228-471d-b6e9-2f3212a7bbf9 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:24.794367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7377dc6-a228-471d-b6e9-2f3212a7bbf9 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:24.800143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09cd36-c458-431d-b3e6-4cab0a227677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:24.800859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09cd36-c458-431d-b3e6-4cab0a227677 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:24.801195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09cd36-c458-431d-b3e6-4cab0a227677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:24.801449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b09cd36-c458-431d-b3e6-4cab0a227677 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:24.802215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09cd36-c458-431d-b3e6-4cab0a227677 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:24.806755 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b09cd36-c458-431d-b3e6-4cab0a227677 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:24.816892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01bfa4-6119-4390-aae7-0282d3c4d027 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:24.817702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01bfa4-6119-4390-aae7-0282d3c4d027 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:24.818081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01bfa4-6119-4390-aae7-0282d3c4d027 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:24.818455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e01bfa4-6119-4390-aae7-0282d3c4d027 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:24.821479 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 924/1848] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:24.822398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09cd36-c458-431d-b3e6-4cab0a227677 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:24.830416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01bfa4-6119-4390-aae7-0282d3c4d027 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:24.832109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e01bfa4-6119-4390-aae7-0282d3c4d027 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:24.845727 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 925/1849] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:24.846992 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01bfa4-6119-4390-aae7-0282d3c4d027 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:24.848724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5e01bfa4-6119-4390-aae7-0282d3c4d027 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:24.851644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b09cd36-c458-431d-b3e6-4cab0a227677 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:24.856707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701446e7-e782-4956-b3c4-21c9c4007a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:24.857781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701446e7-e782-4956-b3c4-21c9c4007a59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:24.858142 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701446e7-e782-4956-b3c4-21c9c4007a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:24.858485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701446e7-e782-4956-b3c4-21c9c4007a59 admin admin] PATH_INFO: `/v3/projects/75a351922ccf4414b873a03ca85ba84b/users/2abfac1ab96e46678da133cf7ffe9cd6/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:24.860828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701446e7-e782-4956-b3c4-21c9c4007a59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75a351922ccf4414b873a03ca85ba84b, user_id=2abfac1ab96e46678da133cf7ffe9cd6, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:24.863733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701446e7-e782-4956-b3c4-21c9c4007a59 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:24.865239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e01bfa4-6119-4390-aae7-0282d3c4d027 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:24.870699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418b5de-3210-4154-bc2d-8e0afc4f97c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:24.871878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418b5de-3210-4154-bc2d-8e0afc4f97c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:24.872303 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418b5de-3210-4154-bc2d-8e0afc4f97c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:24.872778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418b5de-3210-4154-bc2d-8e0afc4f97c2 admin admin] PATH_INFO: `/v3/projects/bbd80b12d66442b898679bac15e25263` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:24.874211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418b5de-3210-4154-bc2d-8e0afc4f97c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbd80b12d66442b898679bac15e25263)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:24.876319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418b5de-3210-4154-bc2d-8e0afc4f97c2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:24.877524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 925/1850] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:24 2022] PUT /identity/v3/projects/75a351922ccf4414b873a03ca85ba84b/users/2abfac1ab96e46678da133cf7ffe9cd6/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:24.878416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701446e7-e782-4956-b3c4-21c9c4007a59 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:24.905423 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:24.937673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1418b5de-3210-4154-bc2d-8e0afc4f97c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbd80b12d66442b898679bac15e25263', 'request_id': 'req-1418b5de-3210-4154-bc2d-8e0afc4f97c2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:24.944623 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1418b5de-3210-4154-bc2d-8e0afc4f97c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbd80b12d66442b898679bac15e25263', 'request_id': 'req-1418b5de-3210-4154-bc2d-8e0afc4f97c2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:24.951421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1418b5de-3210-4154-bc2d-8e0afc4f97c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbd80b12d66442b898679bac15e25263', 'request_id': 'req-1418b5de-3210-4154-bc2d-8e0afc4f97c2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:24.962391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701446e7-e782-4956-b3c4-21c9c4007a59 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:24.968103 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1418b5de-3210-4154-bc2d-8e0afc4f97c2 admin admin] The token cache is being invalidate because project bbd80b12d66442b898679bac15e25263 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:24.968392 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1418b5de-3210-4154-bc2d-8e0afc4f97c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:24.969224 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1418b5de-3210-4154-bc2d-8e0afc4f97c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:24.976299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 926/1851] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:24 2022] DELETE /identity/v3/projects/bbd80b12d66442b898679bac15e25263 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:24.976299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5725ff3-37d3-42b3-a272-20ea14b34ac5 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:24.976299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5725ff3-37d3-42b3-a272-20ea14b34ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:24.976299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5725ff3-37d3-42b3-a272-20ea14b34ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:25.002437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a5725ff3-37d3-42b3-a272-20ea14b34ac5 None None] MFA Rules not processed for user `2abfac1ab96e46678da133cf7ffe9cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:25.003790 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a5725ff3-37d3-42b3-a272-20ea14b34ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:25.032993 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292724f-2664-4951-9268-24932e7608b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:25.034090 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292724f-2664-4951-9268-24932e7608b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:25.034593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292724f-2664-4951-9268-24932e7608b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:25.034761 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292724f-2664-4951-9268-24932e7608b1 admin admin] PATH_INFO: `/v3/projects/031b08795b8e4375842c78f305700e61/users/23aa82abbe894f37ae99c5070aeec77b/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:25.036914 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292724f-2664-4951-9268-24932e7608b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=031b08795b8e4375842c78f305700e61, user_id=23aa82abbe894f37ae99c5070aeec77b, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:25.040118 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292724f-2664-4951-9268-24932e7608b1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:25.061496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 926/1852] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:24 2022] PUT /identity/v3/projects/031b08795b8e4375842c78f305700e61/users/23aa82abbe894f37ae99c5070aeec77b/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:25.061496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfbf5b-c07b-4830-b290-65f874b7dc84 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:25.061496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfbf5b-c07b-4830-b290-65f874b7dc84 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:25.061496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfbf5b-c07b-4830-b290-65f874b7dc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:25.083295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-cadfbf5b-c07b-4830-b290-65f874b7dc84 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:25.084521 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cadfbf5b-c07b-4830-b290-65f874b7dc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:25.089638 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:25.109027 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 927/1853] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:24 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:25.109027 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fca41b-5645-47ab-913e-5ba6586aa2b6 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:25.109027 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fca41b-5645-47ab-913e-5ba6586aa2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:25.109027 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fca41b-5645-47ab-913e-5ba6586aa2b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:25.151662 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f6fca41b-5645-47ab-913e-5ba6586aa2b6 None None] MFA Rules not processed for user `23aa82abbe894f37ae99c5070aeec77b`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:25.153443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f6fca41b-5645-47ab-913e-5ba6586aa2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:25.179442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 927/1854] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:25.181855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfbf5b-c07b-4830-b290-65f874b7dc84 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:25.183218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cadfbf5b-c07b-4830-b290-65f874b7dc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:25.265313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 928/1855] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:25.266985 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e677f-3195-4062-981c-83b362374e64 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:25.267104 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e677f-3195-4062-981c-83b362374e64 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:25.267350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e677f-3195-4062-981c-83b362374e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:25.298806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2c8e677f-3195-4062-981c-83b362374e64 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:25.300957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2c8e677f-3195-4062-981c-83b362374e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:25.302213 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfbf5b-c07b-4830-b290-65f874b7dc84 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:25.308849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc39b33-7ef4-4523-b14e-2e146371c46d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:25.309877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc39b33-7ef4-4523-b14e-2e146371c46d service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:25.310405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc39b33-7ef4-4523-b14e-2e146371c46d service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:25.310844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc39b33-7ef4-4523-b14e-2e146371c46d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:25.312993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 929/1856] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:25.314087 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e677f-3195-4062-981c-83b362374e64 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:25.316258 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2c8e677f-3195-4062-981c-83b362374e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:25.333395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e677f-3195-4062-981c-83b362374e64 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:25.342281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dc8a0-93d8-473e-b032-21ce41521f9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:25.343754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dc8a0-93d8-473e-b032-21ce41521f9b service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:25.344251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dc8a0-93d8-473e-b032-21ce41521f9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:25.344714 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19dc8a0-93d8-473e-b032-21ce41521f9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:25.357957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19dc8a0-93d8-473e-b032-21ce41521f9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:25.360206 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19dc8a0-93d8-473e-b032-21ce41521f9b service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:25.364504 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:25.381219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 930/1857] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:25 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:25.382872 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dc8a0-93d8-473e-b032-21ce41521f9b service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:25.404845 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19dc8a0-93d8-473e-b032-21ce41521f9b service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:25.415650 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4186c-0255-4310-b00e-b388b4604b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:25.416553 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4186c-0255-4310-b00e-b388b4604b68 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:25.416836 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4186c-0255-4310-b00e-b388b4604b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:25.417598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4186c-0255-4310-b00e-b388b4604b68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:25.418306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4186c-0255-4310-b00e-b388b4604b68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:14:25.418611 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4186c-0255-4310-b00e-b388b4604b68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:25.422169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4186c-0255-4310-b00e-b388b4604b68 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:25.462797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 931/1858] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:14:25.464015 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4186c-0255-4310-b00e-b388b4604b68 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:25.494116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4186c-0255-4310-b00e-b388b4604b68 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:25.500124 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc39b33-7ef4-4523-b14e-2e146371c46d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:25.501745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc39b33-7ef4-4523-b14e-2e146371c46d service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:25.502467 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04342d-3ca9-46f7-a1fd-0cca3f37bd7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:25.503315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04342d-3ca9-46f7-a1fd-0cca3f37bd7a admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:25.503748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04342d-3ca9-46f7-a1fd-0cca3f37bd7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:25.504180 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d04342d-3ca9-46f7-a1fd-0cca3f37bd7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:25.505065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04342d-3ca9-46f7-a1fd-0cca3f37bd7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:25.507667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d04342d-3ca9-46f7-a1fd-0cca3f37bd7a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:25.528862 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 928/1859] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:25.529922 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc39b33-7ef4-4523-b14e-2e146371c46d service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:25.549804 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc39b33-7ef4-4523-b14e-2e146371c46d service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:25.562330 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d640314-00fd-45ad-bd45-5746294984af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:25.563271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d640314-00fd-45ad-bd45-5746294984af service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:25.563942 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d640314-00fd-45ad-bd45-5746294984af service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:25.564220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d640314-00fd-45ad-bd45-5746294984af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:25.568517 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8d640314-00fd-45ad-bd45-5746294984af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:25.577284 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 932/1860] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:25 2022] POST /identity/v3/projects => generated 381 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:25.579649 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04342d-3ca9-46f7-a1fd-0cca3f37bd7a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:25.709293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d640314-00fd-45ad-bd45-5746294984af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:25.712678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d04342d-3ca9-46f7-a1fd-0cca3f37bd7a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:25.714657 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d640314-00fd-45ad-bd45-5746294984af service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:25.718586 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fecff9-32b4-4a03-85ee-7c7c1c37e399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:25.729250 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fecff9-32b4-4a03-85ee-7c7c1c37e399 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:25.732032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fecff9-32b4-4a03-85ee-7c7c1c37e399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:25.733720 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fecff9-32b4-4a03-85ee-7c7c1c37e399 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:25.736364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fecff9-32b4-4a03-85ee-7c7c1c37e399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:25.740056 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 929/1861] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:25.742058 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fecff9-32b4-4a03-85ee-7c7c1c37e399 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:25.754320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d640314-00fd-45ad-bd45-5746294984af service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:25.777600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d640314-00fd-45ad-bd45-5746294984af service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:25.790925 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 933/1862] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:25 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:25.796758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fecff9-32b4-4a03-85ee-7c7c1c37e399 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:25.823323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fecff9-32b4-4a03-85ee-7c7c1c37e399 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:25.839522 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44169c62-0890-429b-8e2c-a01044dedc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:25.840383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44169c62-0890-429b-8e2c-a01044dedc1a admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:25.840768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44169c62-0890-429b-8e2c-a01044dedc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:25.841232 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44169c62-0890-429b-8e2c-a01044dedc1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:25.842938 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44169c62-0890-429b-8e2c-a01044dedc1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:25.847149 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44169c62-0890-429b-8e2c-a01044dedc1a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:25.865111 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 934/1863] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:25.871650 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44169c62-0890-429b-8e2c-a01044dedc1a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:25.880121 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:25.886369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44169c62-0890-429b-8e2c-a01044dedc1a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:25.893703 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461c1ec-324a-4e6e-9ded-d5f07db044c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:25.894838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461c1ec-324a-4e6e-9ded-d5f07db044c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:25.895877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461c1ec-324a-4e6e-9ded-d5f07db044c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:25.896366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2461c1ec-324a-4e6e-9ded-d5f07db044c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:25.897284 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461c1ec-324a-4e6e-9ded-d5f07db044c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:25.899291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2461c1ec-324a-4e6e-9ded-d5f07db044c5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:25.908773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352238f9-58e1-4f54-934e-69bd2c299745 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:25.909964 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352238f9-58e1-4f54-934e-69bd2c299745 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:25.910420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352238f9-58e1-4f54-934e-69bd2c299745 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:25.910882 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352238f9-58e1-4f54-934e-69bd2c299745 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:25.920672 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-352238f9-58e1-4f54-934e-69bd2c299745 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:25.930721 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 935/1864] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:25.935299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461c1ec-324a-4e6e-9ded-d5f07db044c5 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:25.963344 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2461c1ec-324a-4e6e-9ded-d5f07db044c5 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:25.980956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb656e01-e662-451f-9537-c83e860bf0ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:25.982582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb656e01-e662-451f-9537-c83e860bf0ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:25.983115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb656e01-e662-451f-9537-c83e860bf0ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:25.985288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb656e01-e662-451f-9537-c83e860bf0ae admin admin] PATH_INFO: `/v3/projects/08d80778f6c445599cf43346143fec39/users/9991fea49b5f43f5916247c2675ea21c/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:26.022024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb656e01-e662-451f-9537-c83e860bf0ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=08d80778f6c445599cf43346143fec39, user_id=9991fea49b5f43f5916247c2675ea21c, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:26.030545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb656e01-e662-451f-9537-c83e860bf0ae admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:26.066151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 936/1865] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:25 2022] PUT /identity/v3/projects/08d80778f6c445599cf43346143fec39/users/9991fea49b5f43f5916247c2675ea21c/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:26.077452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8438f-3c28-4810-9410-20c76091f057 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:26.077739 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8438f-3c28-4810-9410-20c76091f057 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:26.078007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b8438f-3c28-4810-9410-20c76091f057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:26.086702 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352238f9-58e1-4f54-934e-69bd2c299745 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:26.091658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352238f9-58e1-4f54-934e-69bd2c299745 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:26.106344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 930/1866] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:25 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:26.124761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e3b8438f-3c28-4810-9410-20c76091f057 None None] MFA Rules not processed for user `9991fea49b5f43f5916247c2675ea21c`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:26.126815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e3b8438f-3c28-4810-9410-20c76091f057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:26.215365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 937/1867] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:26.219775 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352238f9-58e1-4f54-934e-69bd2c299745 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:26.228377 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:26.237120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352238f9-58e1-4f54-934e-69bd2c299745 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:26.317977 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997589b5-9435-4ebf-8815-c60f1bfe4839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:26.318880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997589b5-9435-4ebf-8815-c60f1bfe4839 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:26.319221 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997589b5-9435-4ebf-8815-c60f1bfe4839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:26.319661 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997589b5-9435-4ebf-8815-c60f1bfe4839 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:26.321764 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997589b5-9435-4ebf-8815-c60f1bfe4839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:26.322638 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997589b5-9435-4ebf-8815-c60f1bfe4839 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:26.388776 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 931/1868] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:26 2022] POST /identity/v3/projects => generated 379 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:26.391116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8438f-3c28-4810-9410-20c76091f057 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:26.415140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b8438f-3c28-4810-9410-20c76091f057 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:26.583459 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b9dc8-f4a9-4efa-88b7-037d65c26183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:26.584397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b9dc8-f4a9-4efa-88b7-037d65c26183 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:26.585580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b9dc8-f4a9-4efa-88b7-037d65c26183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:26.585952 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b9dc8-f4a9-4efa-88b7-037d65c26183 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:26.587289 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b9dc8-f4a9-4efa-88b7-037d65c26183 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:26.591075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997589b5-9435-4ebf-8815-c60f1bfe4839 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:26.593041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-997589b5-9435-4ebf-8815-c60f1bfe4839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:26.595055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b9dc8-f4a9-4efa-88b7-037d65c26183 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:26.618151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997589b5-9435-4ebf-8815-c60f1bfe4839 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:26.630451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:26.632311 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:26.633540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:26.634546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d admin admin] PATH_INFO: `/v3/projects/4cf960aa799c4f3e9ee822e4c746f86b` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:26.636090 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cf960aa799c4f3e9ee822e4c746f86b)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:26.638907 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:26.641023 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 938/1869] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:26 2022] POST /identity/v3/users => generated 296 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:26.641851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b9dc8-f4a9-4efa-88b7-037d65c26183 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:26.666473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b9dc8-f4a9-4efa-88b7-037d65c26183 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:26.684195 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef1cd3-96f4-43d5-b8c3-85172ea58f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:26.685013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef1cd3-96f4-43d5-b8c3-85172ea58f96 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:26.685297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef1cd3-96f4-43d5-b8c3-85172ea58f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:26.685573 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ef1cd3-96f4-43d5-b8c3-85172ea58f96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:26.686329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef1cd3-96f4-43d5-b8c3-85172ea58f96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:26.687943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ef1cd3-96f4-43d5-b8c3-85172ea58f96 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:26.705954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 939/1870] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:26.716875 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cf960aa799c4f3e9ee822e4c746f86b', 'request_id': 'req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:26.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cf960aa799c4f3e9ee822e4c746f86b', 'request_id': 'req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:26.727333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef1cd3-96f4-43d5-b8c3-85172ea58f96 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:26.746949 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ef1cd3-96f4-43d5-b8c3-85172ea58f96 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:26.747535 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cf960aa799c4f3e9ee822e4c746f86b', 'request_id': 'req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:26.767689 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d admin admin] The token cache is being invalidate because project 4cf960aa799c4f3e9ee822e4c746f86b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:26.768065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:26.768896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:26.774060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 932/1871] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:26 2022] DELETE /identity/v3/projects/4cf960aa799c4f3e9ee822e4c746f86b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:26.775393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:26.778460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:26.805669 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:26.835884 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f697c-4834-4ca9-8776-188fb68eb46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:26.836895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f697c-4834-4ca9-8776-188fb68eb46d admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:26.837282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f697c-4834-4ca9-8776-188fb68eb46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:26.837561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f697c-4834-4ca9-8776-188fb68eb46d admin admin] PATH_INFO: `/v3/projects/02a1879a5ddf470cb90b66753deacaf3/users/66a67a1ee2304f0cb4015941e2d5c232/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:26.854195 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f697c-4834-4ca9-8776-188fb68eb46d admin admin] RBAC: Authorizing `identity:create_grant(project_id=02a1879a5ddf470cb90b66753deacaf3, user_id=66a67a1ee2304f0cb4015941e2d5c232, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:26.854710 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a49e8b1-c7d4-4cb1-8750-a85f0032076d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:26.856861 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f697c-4834-4ca9-8776-188fb68eb46d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:26.859979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c5223-6625-4a23-b4a0-e5c25caa409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:26.860875 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c5223-6625-4a23-b4a0-e5c25caa409d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:26.861202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c5223-6625-4a23-b4a0-e5c25caa409d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:26.861452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40c5223-6625-4a23-b4a0-e5c25caa409d admin admin] PATH_INFO: `/v3/users/92e9ab62d9544a3fa94df1e046a6e78f` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:26.862672 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c5223-6625-4a23-b4a0-e5c25caa409d admin admin] RBAC: Authorizing `identity:delete_user(user_id=92e9ab62d9544a3fa94df1e046a6e78f)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:26.864579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40c5223-6625-4a23-b4a0-e5c25caa409d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:26.872601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 940/1872] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:26 2022] PUT /identity/v3/projects/02a1879a5ddf470cb90b66753deacaf3/users/66a67a1ee2304f0cb4015941e2d5c232/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:26.881229 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f697c-4834-4ca9-8776-188fb68eb46d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:26.883559 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-726f697c-4834-4ca9-8776-188fb68eb46d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:26.887287 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:26.938410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f40c5223-6625-4a23-b4a0-e5c25caa409d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92e9ab62d9544a3fa94df1e046a6e78f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:26.939087 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f40c5223-6625-4a23-b4a0-e5c25caa409d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92e9ab62d9544a3fa94df1e046a6e78f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:26.956086 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f40c5223-6625-4a23-b4a0-e5c25caa409d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92e9ab62d9544a3fa94df1e046a6e78f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:26.964133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f40c5223-6625-4a23-b4a0-e5c25caa409d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92e9ab62d9544a3fa94df1e046a6e78f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:26.973004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f40c5223-6625-4a23-b4a0-e5c25caa409d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92e9ab62d9544a3fa94df1e046a6e78f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:26.986797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f697c-4834-4ca9-8776-188fb68eb46d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:26.988686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 933/1873] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:26 2022] DELETE /identity/v3/users/92e9ab62d9544a3fa94df1e046a6e78f => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:27.081225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959df44-9ab1-4de8-b554-a3459247ea76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:27.081886 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959df44-9ab1-4de8-b554-a3459247ea76 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:27.083216 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959df44-9ab1-4de8-b554-a3459247ea76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:27.083216 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959df44-9ab1-4de8-b554-a3459247ea76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:27.083431 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959df44-9ab1-4de8-b554-a3459247ea76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:27.086180 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959df44-9ab1-4de8-b554-a3459247ea76 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:27.104294 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 941/1874] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:27.107918 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c5223-6625-4a23-b4a0-e5c25caa409d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:27.113158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f40c5223-6625-4a23-b4a0-e5c25caa409d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:27.146261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40c5223-6625-4a23-b4a0-e5c25caa409d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:27.160746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9912946-d39b-487b-b418-124607aa0444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:27.172883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9912946-d39b-487b-b418-124607aa0444 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:27.180249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9912946-d39b-487b-b418-124607aa0444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:27.180830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9912946-d39b-487b-b418-124607aa0444 admin admin] PATH_INFO: `/v3/projects/02a1879a5ddf470cb90b66753deacaf3/users/66a67a1ee2304f0cb4015941e2d5c232/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:27.184195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9912946-d39b-487b-b418-124607aa0444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02a1879a5ddf470cb90b66753deacaf3, user_id=66a67a1ee2304f0cb4015941e2d5c232, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:27.187337 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9912946-d39b-487b-b418-124607aa0444 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:27.212985 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 934/1875] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:27 2022] PUT /identity/v3/projects/02a1879a5ddf470cb90b66753deacaf3/users/66a67a1ee2304f0cb4015941e2d5c232/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:27.223839 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9493c-022f-4602-a609-7c6aabf77fcf None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:27.224076 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9493c-022f-4602-a609-7c6aabf77fcf None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:27.224403 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9493c-022f-4602-a609-7c6aabf77fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:27.279858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a3c9493c-022f-4602-a609-7c6aabf77fcf None None] MFA Rules not processed for user `66a67a1ee2304f0cb4015941e2d5c232`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:27.281081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a3c9493c-022f-4602-a609-7c6aabf77fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:27.362179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:27.389981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 942/1876] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:27.405087 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9912946-d39b-487b-b418-124607aa0444 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:27.407808 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d9912946-d39b-487b-b418-124607aa0444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:27.451958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9493c-022f-4602-a609-7c6aabf77fcf None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:27.453483 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a3c9493c-022f-4602-a609-7c6aabf77fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:27.552302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9912946-d39b-487b-b418-124607aa0444 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:27.560070 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9493c-022f-4602-a609-7c6aabf77fcf None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:27.562000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef7a7c9-cc84-4a6a-858e-635e98d971f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:27.564401 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7a7c9-cc84-4a6a-858e-635e98d971f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:27.564782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7a7c9-cc84-4a6a-858e-635e98d971f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:27.565799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef7a7c9-cc84-4a6a-858e-635e98d971f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:27.565979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be2f7fb-bc35-45de-8d3a-83aaae432745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:27.566889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2f7fb-bc35-45de-8d3a-83aaae432745 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:27.567174 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2f7fb-bc35-45de-8d3a-83aaae432745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:27.567451 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be2f7fb-bc35-45de-8d3a-83aaae432745 admin admin] PATH_INFO: `/v3/projects/b4c2912dfbb5457ca2a784d60f306866` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:27.569115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2f7fb-bc35-45de-8d3a-83aaae432745 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4c2912dfbb5457ca2a784d60f306866)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:27.571280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be2f7fb-bc35-45de-8d3a-83aaae432745 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:27.591944 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef7a7c9-cc84-4a6a-858e-635e98d971f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:27.593470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef7a7c9-cc84-4a6a-858e-635e98d971f0 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:27.608807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 935/1877] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:27 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:27.680233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9be2f7fb-bc35-45de-8d3a-83aaae432745 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4c2912dfbb5457ca2a784d60f306866', 'request_id': 'req-9be2f7fb-bc35-45de-8d3a-83aaae432745'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:27.690911 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9be2f7fb-bc35-45de-8d3a-83aaae432745 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4c2912dfbb5457ca2a784d60f306866', 'request_id': 'req-9be2f7fb-bc35-45de-8d3a-83aaae432745'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:27.716445 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9be2f7fb-bc35-45de-8d3a-83aaae432745 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4c2912dfbb5457ca2a784d60f306866', 'request_id': 'req-9be2f7fb-bc35-45de-8d3a-83aaae432745'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:27.737883 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9be2f7fb-bc35-45de-8d3a-83aaae432745 admin admin] The token cache is being invalidate because project b4c2912dfbb5457ca2a784d60f306866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:27.738324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9be2f7fb-bc35-45de-8d3a-83aaae432745 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:27.739159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9be2f7fb-bc35-45de-8d3a-83aaae432745 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:27.744990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 943/1878] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:27 2022] DELETE /identity/v3/projects/b4c2912dfbb5457ca2a784d60f306866 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:27.749884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8109c-07e3-4281-bb47-688643159890 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:27.750239 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8109c-07e3-4281-bb47-688643159890 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:27.750577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee8109c-07e3-4281-bb47-688643159890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:27.793812 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-5ee8109c-07e3-4281-bb47-688643159890 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:27.800052 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5ee8109c-07e3-4281-bb47-688643159890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:27.997999 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b525ba6-e770-4ffc-a5a7-5f909ae1a0b0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:27.998139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 936/1879] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:27.999398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b525ba6-e770-4ffc-a5a7-5f909ae1a0b0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:28.000150 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b525ba6-e770-4ffc-a5a7-5f909ae1a0b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:28.032984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0b525ba6-e770-4ffc-a5a7-5f909ae1a0b0 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:28.034905 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0b525ba6-e770-4ffc-a5a7-5f909ae1a0b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:28.052074 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 944/1880] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:28.058359 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8109c-07e3-4281-bb47-688643159890 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:28.086206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee8109c-07e3-4281-bb47-688643159890 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:28.095335 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5bd86-97fa-4877-8ae2-92d0d2881ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:28.097560 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5bd86-97fa-4877-8ae2-92d0d2881ace admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:28.098907 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5bd86-97fa-4877-8ae2-92d0d2881ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:28.099416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f5bd86-97fa-4877-8ae2-92d0d2881ace admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:28.101152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5bd86-97fa-4877-8ae2-92d0d2881ace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:14:28.101705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5bd86-97fa-4877-8ae2-92d0d2881ace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:28.103579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f5bd86-97fa-4877-8ae2-92d0d2881ace admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:28.126069 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 937/1881] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:14:28.128017 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b525ba6-e770-4ffc-a5a7-5f909ae1a0b0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:28.144612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b525ba6-e770-4ffc-a5a7-5f909ae1a0b0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:28.164013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192608a-b335-4e79-81aa-7b31b27f7b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:28.166805 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192608a-b335-4e79-81aa-7b31b27f7b9c admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:28.167207 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192608a-b335-4e79-81aa-7b31b27f7b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:28.168172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192608a-b335-4e79-81aa-7b31b27f7b9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:28.169770 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a192608a-b335-4e79-81aa-7b31b27f7b9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:28.193669 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a192608a-b335-4e79-81aa-7b31b27f7b9c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:28.307624 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5bd86-97fa-4877-8ae2-92d0d2881ace admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:28.310681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 945/1882] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:28 2022] POST /identity/v3/projects => generated 385 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:28.329027 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f5bd86-97fa-4877-8ae2-92d0d2881ace admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:28.337440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:28.434739 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192608a-b335-4e79-81aa-7b31b27f7b9c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:28.437150 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a192608a-b335-4e79-81aa-7b31b27f7b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:28.450423 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616fb15-8672-47b7-b851-d572f8d7a8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:28.451521 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616fb15-8672-47b7-b851-d572f8d7a8c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:28.451962 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616fb15-8672-47b7-b851-d572f8d7a8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:28.452496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616fb15-8672-47b7-b851-d572f8d7a8c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:28.454620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616fb15-8672-47b7-b851-d572f8d7a8c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:28.457557 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616fb15-8672-47b7-b851-d572f8d7a8c8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:28.511900 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 938/1883] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:28 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:14:28.515374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa336bb6-429f-4bad-b142-138cc3305213 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:28.517131 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa336bb6-429f-4bad-b142-138cc3305213 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:28.517407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa336bb6-429f-4bad-b142-138cc3305213 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:28.518836 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 939/1884] 10.222.0.61 () {54 vars in 921 bytes} [Thu Jul 14 10:14:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:14:28.521875 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa336bb6-429f-4bad-b142-138cc3305213 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:28.550100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa336bb6-429f-4bad-b142-138cc3305213 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:28.557231 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b195b6f4-cc2c-4a18-9cec-41930df81c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:28.557910 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195b6f4-cc2c-4a18-9cec-41930df81c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:28.558242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195b6f4-cc2c-4a18-9cec-41930df81c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:28.558525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b195b6f4-cc2c-4a18-9cec-41930df81c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:28.559776 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195b6f4-cc2c-4a18-9cec-41930df81c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:28.562668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b195b6f4-cc2c-4a18-9cec-41930df81c96 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:28.585644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 940/1885] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:28.588424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:28.602954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6b117-037c-4d46-afdd-62221b019ac3 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:28.603421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6b117-037c-4d46-afdd-62221b019ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:28.604369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a6b117-037c-4d46-afdd-62221b019ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:28.606542 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192608a-b335-4e79-81aa-7b31b27f7b9c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:28.616286 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360681b9-465f-4bcc-a332-48384e38e139 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a67a1ee2304f0cb4015941e2d5c232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02a1879a5ddf470cb90b66753deacaf3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:28.617572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360681b9-465f-4bcc-a332-48384e38e139 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:28.618223 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360681b9-465f-4bcc-a332-48384e38e139 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:28.618684 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360681b9-465f-4bcc-a332-48384e38e139 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:28.619884 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360681b9-465f-4bcc-a332-48384e38e139 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:14:28.620557 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360681b9-465f-4bcc-a332-48384e38e139 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:28.622548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360681b9-465f-4bcc-a332-48384e38e139 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:28.655475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-82a6b117-037c-4d46-afdd-62221b019ac3 None None] MFA Rules not processed for user `e9f4efa3e2f343c2ac858a277601a7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:28.661823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-82a6b117-037c-4d46-afdd-62221b019ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:28.681912 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 946/1886] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:14:28.689691 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360681b9-465f-4bcc-a332-48384e38e139 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:28.714264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360681b9-465f-4bcc-a332-48384e38e139 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:28.744476 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ed611-07cb-43c3-8620-eba24f11d0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:28.746358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ed611-07cb-43c3-8620-eba24f11d0fc admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:28.746798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ed611-07cb-43c3-8620-eba24f11d0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:28.747221 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2ed611-07cb-43c3-8620-eba24f11d0fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:28.748305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ed611-07cb-43c3-8620-eba24f11d0fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:28.750124 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2ed611-07cb-43c3-8620-eba24f11d0fc admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:28.765217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 947/1887] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:28.766107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ed611-07cb-43c3-8620-eba24f11d0fc admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:28.790457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2ed611-07cb-43c3-8620-eba24f11d0fc admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:28.795412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 941/1888] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Jul 14 10:14:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:28.796298 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6b117-037c-4d46-afdd-62221b019ac3 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:28.807793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a6b117-037c-4d46-afdd-62221b019ac3 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:28.808015 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec4cc1-9c45-44c2-8bca-87afc2aac038 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a67a1ee2304f0cb4015941e2d5c232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02a1879a5ddf470cb90b66753deacaf3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:28.808903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec4cc1-9c45-44c2-8bca-87afc2aac038 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:28.809293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec4cc1-9c45-44c2-8bca-87afc2aac038 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:28.809649 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec4cc1-9c45-44c2-8bca-87afc2aac038 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:28.810453 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec4cc1-9c45-44c2-8bca-87afc2aac038 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:28.817639 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec4cc1-9c45-44c2-8bca-87afc2aac038 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:28.819154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57364374-95e2-4838-969e-9779bf096571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:28.820070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57364374-95e2-4838-969e-9779bf096571 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:28.820355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57364374-95e2-4838-969e-9779bf096571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:28.820634 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57364374-95e2-4838-969e-9779bf096571 admin admin] PATH_INFO: `/v3/projects/fca2649a166b48c292ec86b651eb72e7/users/215d21df8a0f439598e27e6d011e3b9b/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:28.875443 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57364374-95e2-4838-969e-9779bf096571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fca2649a166b48c292ec86b651eb72e7, user_id=215d21df8a0f439598e27e6d011e3b9b, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:28.881928 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57364374-95e2-4838-969e-9779bf096571 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:28.900137 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 948/1889] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:28 2022] POST /identity/v3/projects => generated 354 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:28.903081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec4cc1-9c45-44c2-8bca-87afc2aac038 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:28.921976 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 942/1890] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:28 2022] PUT /identity/v3/projects/fca2649a166b48c292ec86b651eb72e7/users/215d21df8a0f439598e27e6d011e3b9b/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:28.924107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083f4fb-2161-4079-9dec-636c2a7c29f4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:28.924483 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083f4fb-2161-4079-9dec-636c2a7c29f4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:28.925073 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6083f4fb-2161-4079-9dec-636c2a7c29f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:28.963728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6083f4fb-2161-4079-9dec-636c2a7c29f4 None None] MFA Rules not processed for user `215d21df8a0f439598e27e6d011e3b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:28.965327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6083f4fb-2161-4079-9dec-636c2a7c29f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:29.036327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec4cc1-9c45-44c2-8bca-87afc2aac038 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:29.072104 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:29.111677 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 943/1891] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:29.118945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083f4fb-2161-4079-9dec-636c2a7c29f4 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:29.164497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61642a94-a6e6-40fc-856c-c2fc4f60647d tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a67a1ee2304f0cb4015941e2d5c232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02a1879a5ddf470cb90b66753deacaf3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:29.165383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61642a94-a6e6-40fc-856c-c2fc4f60647d tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:29.165739 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61642a94-a6e6-40fc-856c-c2fc4f60647d tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:29.167034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61642a94-a6e6-40fc-856c-c2fc4f60647d tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:29.167885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61642a94-a6e6-40fc-856c-c2fc4f60647d tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:14:29.168274 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61642a94-a6e6-40fc-856c-c2fc4f60647d tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:29.170163 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61642a94-a6e6-40fc-856c-c2fc4f60647d tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:29.194131 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 949/1892] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:14:29.229906 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6083f4fb-2161-4079-9dec-636c2a7c29f4 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:29.243744 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378bb63-2968-4a04-ba4e-efb528683ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:29.244770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378bb63-2968-4a04-ba4e-efb528683ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:29.245463 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378bb63-2968-4a04-ba4e-efb528683ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:29.245908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378bb63-2968-4a04-ba4e-efb528683ea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:29.246778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378bb63-2968-4a04-ba4e-efb528683ea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:29.254370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378bb63-2968-4a04-ba4e-efb528683ea5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:29.260696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61642a94-a6e6-40fc-856c-c2fc4f60647d tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:29.271088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61642a94-a6e6-40fc-856c-c2fc4f60647d tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:29.276595 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c13d3-4b71-4286-b2bd-547f03027ea2 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '66a67a1ee2304f0cb4015941e2d5c232', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02a1879a5ddf470cb90b66753deacaf3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:29.277813 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c13d3-4b71-4286-b2bd-547f03027ea2 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:29.278085 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c13d3-4b71-4286-b2bd-547f03027ea2 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:29.278363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184c13d3-4b71-4286-b2bd-547f03027ea2 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] PATH_INFO: `/v3/projects/451326ad5b9d4073a3b2034d55bc05c7` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:29.280153 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c13d3-4b71-4286-b2bd-547f03027ea2 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] RBAC: Authorizing `identity:delete_project(project_id=451326ad5b9d4073a3b2034d55bc05c7)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:29.282259 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184c13d3-4b71-4286-b2bd-547f03027ea2 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:29.298277 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 944/1893] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:29 2022] POST /identity/v3/projects => generated 385 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:29.302246 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378bb63-2968-4a04-ba4e-efb528683ea5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:29.343695 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:29.345192 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-184c13d3-4b71-4286-b2bd-547f03027ea2 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '451326ad5b9d4073a3b2034d55bc05c7', 'request_id': 'req-184c13d3-4b71-4286-b2bd-547f03027ea2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:29.358244 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-184c13d3-4b71-4286-b2bd-547f03027ea2 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '451326ad5b9d4073a3b2034d55bc05c7', 'request_id': 'req-184c13d3-4b71-4286-b2bd-547f03027ea2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:29.379693 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-184c13d3-4b71-4286-b2bd-547f03027ea2 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '451326ad5b9d4073a3b2034d55bc05c7', 'request_id': 'req-184c13d3-4b71-4286-b2bd-547f03027ea2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:29.396018 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378bb63-2968-4a04-ba4e-efb528683ea5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:29.415514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-184c13d3-4b71-4286-b2bd-547f03027ea2 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] The token cache is being invalidate because project 451326ad5b9d4073a3b2034d55bc05c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:29.417045 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-184c13d3-4b71-4286-b2bd-547f03027ea2 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:29.422779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-184c13d3-4b71-4286-b2bd-547f03027ea2 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:29.430579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 950/1894] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:29 2022] DELETE /identity/v3/projects/451326ad5b9d4073a3b2034d55bc05c7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:29.454495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c13d3-4b71-4286-b2bd-547f03027ea2 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:29.456500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-184c13d3-4b71-4286-b2bd-547f03027ea2 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:29.507426 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a74b2-3e3a-4dbf-bb6b-ac6ddaf7fb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:29.508163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a74b2-3e3a-4dbf-bb6b-ac6ddaf7fb0f admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:29.508452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a74b2-3e3a-4dbf-bb6b-ac6ddaf7fb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:29.508728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a74b2-3e3a-4dbf-bb6b-ac6ddaf7fb0f admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:29.509455 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a74b2-3e3a-4dbf-bb6b-ac6ddaf7fb0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:29.511533 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a74b2-3e3a-4dbf-bb6b-ac6ddaf7fb0f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:29.560693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 945/1895] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:29 2022] POST /identity/v3/users => generated 299 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:29.562051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a74b2-3e3a-4dbf-bb6b-ac6ddaf7fb0f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:29.565623 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-338a74b2-3e3a-4dbf-bb6b-ac6ddaf7fb0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:29.583775 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:29.589944 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a74b2-3e3a-4dbf-bb6b-ac6ddaf7fb0f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:29.602672 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2949621-6e65-443e-98db-2aa2aae69eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:29.605664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2949621-6e65-443e-98db-2aa2aae69eac admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:29.606637 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2949621-6e65-443e-98db-2aa2aae69eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:29.607199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2949621-6e65-443e-98db-2aa2aae69eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:29.609146 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2949621-6e65-443e-98db-2aa2aae69eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:29.612653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2949621-6e65-443e-98db-2aa2aae69eac admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:29.636832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 946/1896] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:29.640715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2949621-6e65-443e-98db-2aa2aae69eac admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:29.663047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184c13d3-4b71-4286-b2bd-547f03027ea2 tempest-VolumeQuotaClassesTest-352821044 tempest-VolumeQuotaClassesTest-352821044-project-admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:29.683276 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b5c85-3cb7-4267-9298-3705e7c9651e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:29.688964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b5c85-3cb7-4267-9298-3705e7c9651e service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:29.689540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2949621-6e65-443e-98db-2aa2aae69eac admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:29.690232 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b5c85-3cb7-4267-9298-3705e7c9651e service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:29.690232 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b5c85-3cb7-4267-9298-3705e7c9651e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:29.696652 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ae6b5c85-3cb7-4267-9298-3705e7c9651e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:29.707873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361276a-21f1-4d48-a8f7-425f189174cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:29.708794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361276a-21f1-4d48-a8f7-425f189174cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:29.709095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361276a-21f1-4d48-a8f7-425f189174cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:29.709371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c361276a-21f1-4d48-a8f7-425f189174cd admin admin] PATH_INFO: `/v3/projects/43ca8f48bd7e48b4a24f980194cc7c89/users/278d4459602c4c0e933991676383ccf5/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:29.732884 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b5c85-3cb7-4267-9298-3705e7c9651e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:29.734872 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6b5c85-3cb7-4267-9298-3705e7c9651e service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:29.746066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c361276a-21f1-4d48-a8f7-425f189174cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ca8f48bd7e48b4a24f980194cc7c89, user_id=278d4459602c4c0e933991676383ccf5, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:29.747299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 951/1897] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:29.760740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c361276a-21f1-4d48-a8f7-425f189174cd admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:29.805953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b5c85-3cb7-4267-9298-3705e7c9651e service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:29.806205 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 947/1898] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:29 2022] PUT /identity/v3/projects/43ca8f48bd7e48b4a24f980194cc7c89/users/278d4459602c4c0e933991676383ccf5/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:29.810857 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361276a-21f1-4d48-a8f7-425f189174cd admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:29.814358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6b5c85-3cb7-4267-9298-3705e7c9651e service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:29.817569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c361276a-21f1-4d48-a8f7-425f189174cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:29.942831 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8704f-f74b-4d21-9e89-322bc9f73f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:29.943942 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c361276a-21f1-4d48-a8f7-425f189174cd admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:29.945074 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8704f-f74b-4d21-9e89-322bc9f73f09 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:29.945522 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8704f-f74b-4d21-9e89-322bc9f73f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:29.945944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8704f-f74b-4d21-9e89-322bc9f73f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:29.947439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8704f-f74b-4d21-9e89-322bc9f73f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:29.949769 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8704f-f74b-4d21-9e89-322bc9f73f09 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:29.950558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c875c37-2056-40ba-9b99-22846b58ff1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:29.951616 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c875c37-2056-40ba-9b99-22846b58ff1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:29.951910 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c875c37-2056-40ba-9b99-22846b58ff1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:29.953342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c875c37-2056-40ba-9b99-22846b58ff1d admin admin] PATH_INFO: `/v3/users/9991fea49b5f43f5916247c2675ea21c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:29.955230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c875c37-2056-40ba-9b99-22846b58ff1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9991fea49b5f43f5916247c2675ea21c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:29.962281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c875c37-2056-40ba-9b99-22846b58ff1d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:29.983462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 952/1899] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:29.988875 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8704f-f74b-4d21-9e89-322bc9f73f09 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:30.023464 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8704f-f74b-4d21-9e89-322bc9f73f09 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:30.032012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73be16b1-7c86-4610-a654-f3b4a729eac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:30.032944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be16b1-7c86-4610-a654-f3b4a729eac0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:30.033358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be16b1-7c86-4610-a654-f3b4a729eac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:30.033527 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73be16b1-7c86-4610-a654-f3b4a729eac0 admin admin] PATH_INFO: `/v3/projects/43ca8f48bd7e48b4a24f980194cc7c89/users/278d4459602c4c0e933991676383ccf5/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:30.038372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be16b1-7c86-4610-a654-f3b4a729eac0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43ca8f48bd7e48b4a24f980194cc7c89, user_id=278d4459602c4c0e933991676383ccf5, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:30.041653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4c875c37-2056-40ba-9b99-22846b58ff1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9991fea49b5f43f5916247c2675ea21c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:30.042108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73be16b1-7c86-4610-a654-f3b4a729eac0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:30.042387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4c875c37-2056-40ba-9b99-22846b58ff1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9991fea49b5f43f5916247c2675ea21c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:30.058676 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4c875c37-2056-40ba-9b99-22846b58ff1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9991fea49b5f43f5916247c2675ea21c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:30.060318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 953/1900] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:29 2022] PUT /identity/v3/projects/43ca8f48bd7e48b4a24f980194cc7c89/users/278d4459602c4c0e933991676383ccf5/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:30.067907 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4c875c37-2056-40ba-9b99-22846b58ff1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9991fea49b5f43f5916247c2675ea21c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:30.073486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aeb0c5-ce07-4260-bb14-ee176e31d951 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:30.073791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aeb0c5-ce07-4260-bb14-ee176e31d951 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:30.075106 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aeb0c5-ce07-4260-bb14-ee176e31d951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:30.078713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4c875c37-2056-40ba-9b99-22846b58ff1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9991fea49b5f43f5916247c2675ea21c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:30.090724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 948/1901] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:29 2022] DELETE /identity/v3/users/9991fea49b5f43f5916247c2675ea21c => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:30.092449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfe438-9afa-44dd-a77c-fb3524308215 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:30.092883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfe438-9afa-44dd-a77c-fb3524308215 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:30.093358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedfe438-9afa-44dd-a77c-fb3524308215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:30.105478 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-19aeb0c5-ce07-4260-bb14-ee176e31d951 None None] MFA Rules not processed for user `278d4459602c4c0e933991676383ccf5`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:30.113146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-19aeb0c5-ce07-4260-bb14-ee176e31d951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:30.118846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-eedfe438-9afa-44dd-a77c-fb3524308215 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:30.120051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-eedfe438-9afa-44dd-a77c-fb3524308215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:30.162651 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:30.190632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:30.213110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 954/1902] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:14:30.214604 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 949/1903] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:30.224812 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aeb0c5-ce07-4260-bb14-ee176e31d951 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:30.225584 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfe438-9afa-44dd-a77c-fb3524308215 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:30.226622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-19aeb0c5-ce07-4260-bb14-ee176e31d951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:30.227000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-eedfe438-9afa-44dd-a77c-fb3524308215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:30.296331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aeb0c5-ce07-4260-bb14-ee176e31d951 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:30.300598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d943404-b204-46df-9c65-4a96ad0474c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:30.301284 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d943404-b204-46df-9c65-4a96ad0474c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:30.301570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d943404-b204-46df-9c65-4a96ad0474c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:30.301892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d943404-b204-46df-9c65-4a96ad0474c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:30.308066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedfe438-9afa-44dd-a77c-fb3524308215 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:30.312337 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeca3ac-74f0-41a6-bef8-5121ae0bce66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:30.313071 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeca3ac-74f0-41a6-bef8-5121ae0bce66 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:30.313199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d943404-b204-46df-9c65-4a96ad0474c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:30.313287 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeca3ac-74f0-41a6-bef8-5121ae0bce66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:30.313536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeca3ac-74f0-41a6-bef8-5121ae0bce66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:30.314458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d943404-b204-46df-9c65-4a96ad0474c7 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:30.324121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeca3ac-74f0-41a6-bef8-5121ae0bce66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:30.324915 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 955/1904] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:30.325748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeca3ac-74f0-41a6-bef8-5121ae0bce66 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:30.336885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 950/1905] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:14:30.695884 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d943404-b204-46df-9c65-4a96ad0474c7 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:30.697579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3d943404-b204-46df-9c65-4a96ad0474c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:30.727329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d943404-b204-46df-9c65-4a96ad0474c7 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:30.739167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc4c77-b227-4a67-896f-1260afd4d6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:30.740005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc4c77-b227-4a67-896f-1260afd4d6b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:30.740290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc4c77-b227-4a67-896f-1260afd4d6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:30.740889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc4c77-b227-4a67-896f-1260afd4d6b2 admin admin] PATH_INFO: `/v3/projects/08d80778f6c445599cf43346143fec39` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:30.742261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc4c77-b227-4a67-896f-1260afd4d6b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08d80778f6c445599cf43346143fec39)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:30.744819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc4c77-b227-4a67-896f-1260afd4d6b2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:30.808299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-35cc4c77-b227-4a67-896f-1260afd4d6b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08d80778f6c445599cf43346143fec39', 'request_id': 'req-35cc4c77-b227-4a67-896f-1260afd4d6b2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:30.815060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-35cc4c77-b227-4a67-896f-1260afd4d6b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08d80778f6c445599cf43346143fec39', 'request_id': 'req-35cc4c77-b227-4a67-896f-1260afd4d6b2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:30.821524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-35cc4c77-b227-4a67-896f-1260afd4d6b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08d80778f6c445599cf43346143fec39', 'request_id': 'req-35cc4c77-b227-4a67-896f-1260afd4d6b2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:30.836679 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-35cc4c77-b227-4a67-896f-1260afd4d6b2 admin admin] The token cache is being invalidate because project 08d80778f6c445599cf43346143fec39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:30.837073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-35cc4c77-b227-4a67-896f-1260afd4d6b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:30.838142 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-35cc4c77-b227-4a67-896f-1260afd4d6b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:30.843519 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 956/1906] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:30 2022] DELETE /identity/v3/projects/08d80778f6c445599cf43346143fec39 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:30.844583 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeca3ac-74f0-41a6-bef8-5121ae0bce66 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:30.864197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeca3ac-74f0-41a6-bef8-5121ae0bce66 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:30.943130 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4038118-7b0d-4a85-a511-8de33295441b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:30.944210 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4038118-7b0d-4a85-a511-8de33295441b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:30.944561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4038118-7b0d-4a85-a511-8de33295441b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:30.944841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4038118-7b0d-4a85-a511-8de33295441b admin admin] PATH_INFO: `/v3/users/66a67a1ee2304f0cb4015941e2d5c232` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:30.946227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4038118-7b0d-4a85-a511-8de33295441b admin admin] RBAC: Authorizing `identity:delete_user(user_id=66a67a1ee2304f0cb4015941e2d5c232)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:30.948382 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4038118-7b0d-4a85-a511-8de33295441b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:31.000856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f4038118-7b0d-4a85-a511-8de33295441b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66a67a1ee2304f0cb4015941e2d5c232'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:31.000994 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f4038118-7b0d-4a85-a511-8de33295441b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66a67a1ee2304f0cb4015941e2d5c232'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:31.014212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f4038118-7b0d-4a85-a511-8de33295441b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66a67a1ee2304f0cb4015941e2d5c232'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:31.021503 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f4038118-7b0d-4a85-a511-8de33295441b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66a67a1ee2304f0cb4015941e2d5c232'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:31.035091 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f4038118-7b0d-4a85-a511-8de33295441b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66a67a1ee2304f0cb4015941e2d5c232'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:31.046319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 951/1907] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:30 2022] DELETE /identity/v3/users/66a67a1ee2304f0cb4015941e2d5c232 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:31.111295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc4c77-b227-4a67-896f-1260afd4d6b2 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:31.113150 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-35cc4c77-b227-4a67-896f-1260afd4d6b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:31.126754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc4c77-b227-4a67-896f-1260afd4d6b2 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:31.191409 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:31.202729 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04e420-846a-47ac-99c4-d7ad8976c8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:31.203621 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04e420-846a-47ac-99c4-d7ad8976c8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:31.204424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04e420-846a-47ac-99c4-d7ad8976c8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:31.205100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f04e420-846a-47ac-99c4-d7ad8976c8fa admin admin] PATH_INFO: `/v3/users/215d21df8a0f439598e27e6d011e3b9b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:31.206917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04e420-846a-47ac-99c4-d7ad8976c8fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=215d21df8a0f439598e27e6d011e3b9b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:31.209116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f04e420-846a-47ac-99c4-d7ad8976c8fa admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:31.276269 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7f04e420-846a-47ac-99c4-d7ad8976c8fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '215d21df8a0f439598e27e6d011e3b9b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:31.277057 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7f04e420-846a-47ac-99c4-d7ad8976c8fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '215d21df8a0f439598e27e6d011e3b9b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:31.289788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7f04e420-846a-47ac-99c4-d7ad8976c8fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '215d21df8a0f439598e27e6d011e3b9b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:31.296598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7f04e420-846a-47ac-99c4-d7ad8976c8fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '215d21df8a0f439598e27e6d011e3b9b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:31.304496 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7f04e420-846a-47ac-99c4-d7ad8976c8fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '215d21df8a0f439598e27e6d011e3b9b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:31.310827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 957/1908] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:31 2022] DELETE /identity/v3/users/215d21df8a0f439598e27e6d011e3b9b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:14:31.315410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44985953-083a-4329-96cb-61f5f924df0a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:31.315883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44985953-083a-4329-96cb-61f5f924df0a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:31.315883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44985953-083a-4329-96cb-61f5f924df0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:31.334139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-44985953-083a-4329-96cb-61f5f924df0a None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:31.336548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-44985953-083a-4329-96cb-61f5f924df0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:31.342613 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:31.356762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04e420-846a-47ac-99c4-d7ad8976c8fa admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:31.360625 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7f04e420-846a-47ac-99c4-d7ad8976c8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:31.425175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 952/1909] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:14:31.436661 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44985953-083a-4329-96cb-61f5f924df0a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:31.438034 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-44985953-083a-4329-96cb-61f5f924df0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:31.448722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f04e420-846a-47ac-99c4-d7ad8976c8fa admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:31.453679 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef5d15-5c9f-4951-bab5-e4d17f9047b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:31.454625 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef5d15-5c9f-4951-bab5-e4d17f9047b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:31.454915 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef5d15-5c9f-4951-bab5-e4d17f9047b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:31.455319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef5d15-5c9f-4951-bab5-e4d17f9047b5 admin admin] PATH_INFO: `/v3/projects/02a1879a5ddf470cb90b66753deacaf3` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:31.456791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef5d15-5c9f-4951-bab5-e4d17f9047b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02a1879a5ddf470cb90b66753deacaf3)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:31.458896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfef5d15-5c9f-4951-bab5-e4d17f9047b5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:31.500789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44985953-083a-4329-96cb-61f5f924df0a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:31.505348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bfef5d15-5c9f-4951-bab5-e4d17f9047b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02a1879a5ddf470cb90b66753deacaf3', 'request_id': 'req-bfef5d15-5c9f-4951-bab5-e4d17f9047b5'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:31.512269 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bfef5d15-5c9f-4951-bab5-e4d17f9047b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02a1879a5ddf470cb90b66753deacaf3', 'request_id': 'req-bfef5d15-5c9f-4951-bab5-e4d17f9047b5'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:31.517747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bfef5d15-5c9f-4951-bab5-e4d17f9047b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02a1879a5ddf470cb90b66753deacaf3', 'request_id': 'req-bfef5d15-5c9f-4951-bab5-e4d17f9047b5'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:31.531790 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bfef5d15-5c9f-4951-bab5-e4d17f9047b5 admin admin] The token cache is being invalidate because project 02a1879a5ddf470cb90b66753deacaf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:31.532165 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bfef5d15-5c9f-4951-bab5-e4d17f9047b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:31.533033 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bfef5d15-5c9f-4951-bab5-e4d17f9047b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:31.534898 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 958/1910] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:31 2022] DELETE /identity/v3/projects/02a1879a5ddf470cb90b66753deacaf3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:31.539580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970a288-810b-41ca-8e0b-7829a5e07cd6 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:31.539865 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970a288-810b-41ca-8e0b-7829a5e07cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:31.540140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970a288-810b-41ca-8e0b-7829a5e07cd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:31.548205 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:31.552244 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333456e-bac2-45ff-9c53-b17fde3dda47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:31.553011 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333456e-bac2-45ff-9c53-b17fde3dda47 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:31.553877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333456e-bac2-45ff-9c53-b17fde3dda47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:31.554318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333456e-bac2-45ff-9c53-b17fde3dda47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:31.556350 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f333456e-bac2-45ff-9c53-b17fde3dda47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:31.561392 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3970a288-810b-41ca-8e0b-7829a5e07cd6 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:31.563858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3970a288-810b-41ca-8e0b-7829a5e07cd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:31.621883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333456e-bac2-45ff-9c53-b17fde3dda47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:31.623224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333456e-bac2-45ff-9c53-b17fde3dda47 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:31.624433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 959/1911] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:14:31.632648 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea78e33-d21f-4448-8022-8d931c96391b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:31.632959 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea78e33-d21f-4448-8022-8d931c96391b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:31.633237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea78e33-d21f-4448-8022-8d931c96391b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:31.633751 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 953/1912] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:14:31.655645 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-aea78e33-d21f-4448-8022-8d931c96391b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:31.656844 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-aea78e33-d21f-4448-8022-8d931c96391b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:31.664860 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 960/1913] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:14:31.669681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333456e-bac2-45ff-9c53-b17fde3dda47 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:31.679701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333456e-bac2-45ff-9c53-b17fde3dda47 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:31.684607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09710a9-1d62-4a5d-bcdc-5be863fc46a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:31.685635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09710a9-1d62-4a5d-bcdc-5be863fc46a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:31.685635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09710a9-1d62-4a5d-bcdc-5be863fc46a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:31.685893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09710a9-1d62-4a5d-bcdc-5be863fc46a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:31.686742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09710a9-1d62-4a5d-bcdc-5be863fc46a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:14:31.687051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09710a9-1d62-4a5d-bcdc-5be863fc46a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:31.688681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09710a9-1d62-4a5d-bcdc-5be863fc46a1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:31.703178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 954/1914] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:14:31.710128 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea78e33-d21f-4448-8022-8d931c96391b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:31.721100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea78e33-d21f-4448-8022-8d931c96391b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:31.726121 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6080de-53bf-4077-8d8e-d9aa469a5192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:31.726838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6080de-53bf-4077-8d8e-d9aa469a5192 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:31.727120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6080de-53bf-4077-8d8e-d9aa469a5192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:31.727389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6080de-53bf-4077-8d8e-d9aa469a5192 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:31.728080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6080de-53bf-4077-8d8e-d9aa469a5192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:31.730141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6080de-53bf-4077-8d8e-d9aa469a5192 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:31.759005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 961/1915] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:31 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:14:31.763601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09710a9-1d62-4a5d-bcdc-5be863fc46a1 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:31.778284 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:31.780912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09710a9-1d62-4a5d-bcdc-5be863fc46a1 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:31.852754 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3302c68-8fa7-4528-9a70-eb534a1f3d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:31.853771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3302c68-8fa7-4528-9a70-eb534a1f3d68 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:31.853771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3302c68-8fa7-4528-9a70-eb534a1f3d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:31.854017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3302c68-8fa7-4528-9a70-eb534a1f3d68 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:31.854762 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3302c68-8fa7-4528-9a70-eb534a1f3d68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:31.856902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3302c68-8fa7-4528-9a70-eb534a1f3d68 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:31.886442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 955/1916] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:31 2022] POST /identity/v3/users => generated 295 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:14:31.891405 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6080de-53bf-4077-8d8e-d9aa469a5192 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:31.905553 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6080de-53bf-4077-8d8e-d9aa469a5192 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:31.911547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42d492-b882-4b57-8f23-53e94c347934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:31.912469 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42d492-b882-4b57-8f23-53e94c347934 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:31.913455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42d492-b882-4b57-8f23-53e94c347934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:31.913736 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa42d492-b882-4b57-8f23-53e94c347934 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:31.915165 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa42d492-b882-4b57-8f23-53e94c347934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:31.916953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa42d492-b882-4b57-8f23-53e94c347934 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:31.935354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 962/1917] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:31.945318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3302c68-8fa7-4528-9a70-eb534a1f3d68 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:31.957606 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3302c68-8fa7-4528-9a70-eb534a1f3d68 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:31.962921 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717b556-58a3-45dd-9d9a-3f3e3eabefd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:31.963594 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717b556-58a3-45dd-9d9a-3f3e3eabefd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:31.963874 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717b556-58a3-45dd-9d9a-3f3e3eabefd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:31.964143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f717b556-58a3-45dd-9d9a-3f3e3eabefd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:31.965317 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717b556-58a3-45dd-9d9a-3f3e3eabefd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:31.967051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f717b556-58a3-45dd-9d9a-3f3e3eabefd4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:31.986794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 956/1918] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:31.991769 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42d492-b882-4b57-8f23-53e94c347934 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:32.005182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa42d492-b882-4b57-8f23-53e94c347934 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:32.010055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf564b04-1519-4a45-9af2-a100d88ce57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:32.010963 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf564b04-1519-4a45-9af2-a100d88ce57f admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:32.011249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf564b04-1519-4a45-9af2-a100d88ce57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:32.011757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf564b04-1519-4a45-9af2-a100d88ce57f admin admin] PATH_INFO: `/v3/projects/7d38092cafa94ce2838aef4ae7968c1f/users/2536887cef034f189be889a0e9c35a16/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:32.032167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf564b04-1519-4a45-9af2-a100d88ce57f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d38092cafa94ce2838aef4ae7968c1f, user_id=2536887cef034f189be889a0e9c35a16, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:32.035426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf564b04-1519-4a45-9af2-a100d88ce57f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:32.055677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 963/1919] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:31 2022] PUT /identity/v3/projects/7d38092cafa94ce2838aef4ae7968c1f/users/2536887cef034f189be889a0e9c35a16/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:32.067833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717b556-58a3-45dd-9d9a-3f3e3eabefd4 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:32.070225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f717b556-58a3-45dd-9d9a-3f3e3eabefd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:32.074019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb7201-995d-447f-97f0-47ad0c51146b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:32.074331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb7201-995d-447f-97f0-47ad0c51146b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:32.074569 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb7201-995d-447f-97f0-47ad0c51146b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:32.110998 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f717b556-58a3-45dd-9d9a-3f3e3eabefd4 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:32.151980 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2bcb7201-995d-447f-97f0-47ad0c51146b None None] MFA Rules not processed for user `2536887cef034f189be889a0e9c35a16`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:32.153498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2bcb7201-995d-447f-97f0-47ad0c51146b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:32.265316 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 964/1920] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:32.270494 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb7201-995d-447f-97f0-47ad0c51146b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:32.273782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b139fec-fe4c-46fb-a2d0-369affd6842c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:32.277076 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b139fec-fe4c-46fb-a2d0-369affd6842c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:32.278804 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b139fec-fe4c-46fb-a2d0-369affd6842c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:32.280829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b139fec-fe4c-46fb-a2d0-369affd6842c admin admin] PATH_INFO: `/v3/projects/fca2649a166b48c292ec86b651eb72e7` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:32.283689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcb7201-995d-447f-97f0-47ad0c51146b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:32.284982 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b139fec-fe4c-46fb-a2d0-369affd6842c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fca2649a166b48c292ec86b651eb72e7)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:32.290555 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbd071-de00-4e2e-af00-4ce5c9df57c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:32.290892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbd071-de00-4e2e-af00-4ce5c9df57c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:32.291531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbd071-de00-4e2e-af00-4ce5c9df57c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:32.291531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfbd071-de00-4e2e-af00-4ce5c9df57c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:32.292487 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b139fec-fe4c-46fb-a2d0-369affd6842c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:32.292619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbd071-de00-4e2e-af00-4ce5c9df57c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:32.295949 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfbd071-de00-4e2e-af00-4ce5c9df57c0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:32.343937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 965/1921] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:32 2022] POST /identity/v3/projects => generated 387 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:14:32.347351 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbd071-de00-4e2e-af00-4ce5c9df57c0 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:32.376116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:32.425844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8b139fec-fe4c-46fb-a2d0-369affd6842c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fca2649a166b48c292ec86b651eb72e7', 'request_id': 'req-8b139fec-fe4c-46fb-a2d0-369affd6842c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:32.442353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8b139fec-fe4c-46fb-a2d0-369affd6842c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fca2649a166b48c292ec86b651eb72e7', 'request_id': 'req-8b139fec-fe4c-46fb-a2d0-369affd6842c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:32.467304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8b139fec-fe4c-46fb-a2d0-369affd6842c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fca2649a166b48c292ec86b651eb72e7', 'request_id': 'req-8b139fec-fe4c-46fb-a2d0-369affd6842c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:32.483724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8b139fec-fe4c-46fb-a2d0-369affd6842c admin admin] The token cache is being invalidate because project fca2649a166b48c292ec86b651eb72e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:32.484153 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8b139fec-fe4c-46fb-a2d0-369affd6842c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:32.484894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-8b139fec-fe4c-46fb-a2d0-369affd6842c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:32.487836 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 957/1922] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:32 2022] DELETE /identity/v3/projects/fca2649a166b48c292ec86b651eb72e7 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:32.492406 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b139fec-fe4c-46fb-a2d0-369affd6842c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:32.495570 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8b139fec-fe4c-46fb-a2d0-369affd6842c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:32.517179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfbd071-de00-4e2e-af00-4ce5c9df57c0 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:32.554937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:32.618792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e194a5-5188-4931-8ae7-36b6f17147ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:32.619740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e194a5-5188-4931-8ae7-36b6f17147ec admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:32.620030 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e194a5-5188-4931-8ae7-36b6f17147ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:32.620308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e194a5-5188-4931-8ae7-36b6f17147ec admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:32.621161 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e194a5-5188-4931-8ae7-36b6f17147ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:32.623197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e194a5-5188-4931-8ae7-36b6f17147ec admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:32.658525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 966/1923] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:32 2022] POST /identity/v3/users => generated 300 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:32.660708 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e194a5-5188-4931-8ae7-36b6f17147ec admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:32.663794 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-52e194a5-5188-4931-8ae7-36b6f17147ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:32.671209 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b139fec-fe4c-46fb-a2d0-369affd6842c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:32.685758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e194a5-5188-4931-8ae7-36b6f17147ec admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:32.687024 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f8508-9b87-4836-9427-520b6ee03ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:32.688603 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8508-9b87-4836-9427-520b6ee03ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:32.689292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8508-9b87-4836-9427-520b6ee03ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:32.689448 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8508-9b87-4836-9427-520b6ee03ccd admin admin] PATH_INFO: `/v3/users/278d4459602c4c0e933991676383ccf5` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:32.691269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f8508-9b87-4836-9427-520b6ee03ccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=278d4459602c4c0e933991676383ccf5)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:32.694636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a60d5f-11bb-4cb0-b844-e525b047f3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:32.695732 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a60d5f-11bb-4cb0-b844-e525b047f3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:32.695880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f8508-9b87-4836-9427-520b6ee03ccd admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:32.696316 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a60d5f-11bb-4cb0-b844-e525b047f3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:32.696823 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a60d5f-11bb-4cb0-b844-e525b047f3c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:32.698516 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a60d5f-11bb-4cb0-b844-e525b047f3c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:32.701283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a60d5f-11bb-4cb0-b844-e525b047f3c9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:32.720248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 967/1924] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:32.723314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a60d5f-11bb-4cb0-b844-e525b047f3c9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:32.737248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a60d5f-11bb-4cb0-b844-e525b047f3c9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:32.747308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e0046-33e6-49fb-820f-76568b6969de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:32.748637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e0046-33e6-49fb-820f-76568b6969de admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:32.749296 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e0046-33e6-49fb-820f-76568b6969de admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:32.749676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e0046-33e6-49fb-820f-76568b6969de admin admin] PATH_INFO: `/v3/projects/ca443abe0e44403086a7dab7ef3f379a/users/8b33138c7f42495787adb6eb8f87ef24/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:32.775987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e0046-33e6-49fb-820f-76568b6969de admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca443abe0e44403086a7dab7ef3f379a, user_id=8b33138c7f42495787adb6eb8f87ef24, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:32.779268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2e0046-33e6-49fb-820f-76568b6969de admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:32.795530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 968/1925] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:32 2022] PUT /identity/v3/projects/ca443abe0e44403086a7dab7ef3f379a/users/8b33138c7f42495787adb6eb8f87ef24/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:32.799495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e0046-33e6-49fb-820f-76568b6969de admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:32.804641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-220f8508-9b87-4836-9427-520b6ee03ccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '278d4459602c4c0e933991676383ccf5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:32.806332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-220f8508-9b87-4836-9427-520b6ee03ccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '278d4459602c4c0e933991676383ccf5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:32.821150 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-220f8508-9b87-4836-9427-520b6ee03ccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '278d4459602c4c0e933991676383ccf5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:32.830908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-220f8508-9b87-4836-9427-520b6ee03ccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '278d4459602c4c0e933991676383ccf5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:32.842692 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-220f8508-9b87-4836-9427-520b6ee03ccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '278d4459602c4c0e933991676383ccf5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:32.851986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 958/1926] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:32 2022] DELETE /identity/v3/users/278d4459602c4c0e933991676383ccf5 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:32.878615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2e0046-33e6-49fb-820f-76568b6969de admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:32.903995 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:32.980879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe38fc-4b86-4534-b770-436b951087cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:32.981598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe38fc-4b86-4534-b770-436b951087cf admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:32.981882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe38fc-4b86-4534-b770-436b951087cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:32.985145 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe38fc-4b86-4534-b770-436b951087cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:32.986016 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe38fc-4b86-4534-b770-436b951087cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:32.989487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babe38fc-4b86-4534-b770-436b951087cf admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:33.012120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 969/1927] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:33.015889 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f8508-9b87-4836-9427-520b6ee03ccd admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:33.017461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-220f8508-9b87-4836-9427-520b6ee03ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:33.039533 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f8508-9b87-4836-9427-520b6ee03ccd admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:33.050926 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad130f5-9320-4995-8f21-827f5c9a983b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:33.051455 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad130f5-9320-4995-8f21-827f5c9a983b admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:33.051660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad130f5-9320-4995-8f21-827f5c9a983b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:33.052466 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad130f5-9320-4995-8f21-827f5c9a983b admin admin] PATH_INFO: `/v3/projects/ca443abe0e44403086a7dab7ef3f379a/users/8b33138c7f42495787adb6eb8f87ef24/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:33.056759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad130f5-9320-4995-8f21-827f5c9a983b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca443abe0e44403086a7dab7ef3f379a, user_id=8b33138c7f42495787adb6eb8f87ef24, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:33.062907 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad130f5-9320-4995-8f21-827f5c9a983b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:33.091657 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 959/1928] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:33 2022] PUT /identity/v3/projects/ca443abe0e44403086a7dab7ef3f379a/users/8b33138c7f42495787adb6eb8f87ef24/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:33.104136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd725a1-c11e-4738-aa41-9df38d2fd083 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:33.104428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd725a1-c11e-4738-aa41-9df38d2fd083 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:33.104728 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd725a1-c11e-4738-aa41-9df38d2fd083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:33.163237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8cd725a1-c11e-4738-aa41-9df38d2fd083 None None] MFA Rules not processed for user `8b33138c7f42495787adb6eb8f87ef24`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:33.164481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8cd725a1-c11e-4738-aa41-9df38d2fd083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:33.245942 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad130f5-9320-4995-8f21-827f5c9a983b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:33.247806 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8ad130f5-9320-4995-8f21-827f5c9a983b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:33.329575 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 970/1929] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:33.330953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd725a1-c11e-4738-aa41-9df38d2fd083 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:33.332835 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8cd725a1-c11e-4738-aa41-9df38d2fd083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:33.387278 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad130f5-9320-4995-8f21-827f5c9a983b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:33.393254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3675b-bf28-430a-bd14-6036d0b30865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:33.394279 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3675b-bf28-430a-bd14-6036d0b30865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:33.394987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3675b-bf28-430a-bd14-6036d0b30865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:33.395109 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a3675b-bf28-430a-bd14-6036d0b30865 admin admin] PATH_INFO: `/v3/users/57e492f57e304038a184dd6f4dbec3d7` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:33.396390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3675b-bf28-430a-bd14-6036d0b30865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57e492f57e304038a184dd6f4dbec3d7)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:33.398687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a3675b-bf28-430a-bd14-6036d0b30865 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:33.429257 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:33.431862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd725a1-c11e-4738-aa41-9df38d2fd083 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:33.442462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1079f4-a9a6-4984-931b-1e764b2c98c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:33.443504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1079f4-a9a6-4984-931b-1e764b2c98c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:33.444146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1079f4-a9a6-4984-931b-1e764b2c98c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:33.444419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1079f4-a9a6-4984-931b-1e764b2c98c6 admin admin] PATH_INFO: `/v3/projects/43ca8f48bd7e48b4a24f980194cc7c89` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:33.446047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1079f4-a9a6-4984-931b-1e764b2c98c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43ca8f48bd7e48b4a24f980194cc7c89)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:33.448140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1079f4-a9a6-4984-931b-1e764b2c98c6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:33.465929 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c5a3675b-bf28-430a-bd14-6036d0b30865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57e492f57e304038a184dd6f4dbec3d7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:33.467179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c5a3675b-bf28-430a-bd14-6036d0b30865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57e492f57e304038a184dd6f4dbec3d7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:33.485758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c5a3675b-bf28-430a-bd14-6036d0b30865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57e492f57e304038a184dd6f4dbec3d7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:33.498777 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c5a3675b-bf28-430a-bd14-6036d0b30865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57e492f57e304038a184dd6f4dbec3d7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:33.507635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c5a3675b-bf28-430a-bd14-6036d0b30865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57e492f57e304038a184dd6f4dbec3d7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:33.511284 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:33.525143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 960/1930] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:33 2022] DELETE /identity/v3/users/57e492f57e304038a184dd6f4dbec3d7 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:33.525143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3675b-bf28-430a-bd14-6036d0b30865 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:33.527768 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c5a3675b-bf28-430a-bd14-6036d0b30865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:33.530173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5f1079f4-a9a6-4984-931b-1e764b2c98c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43ca8f48bd7e48b4a24f980194cc7c89', 'request_id': 'req-5f1079f4-a9a6-4984-931b-1e764b2c98c6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:33.542556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5f1079f4-a9a6-4984-931b-1e764b2c98c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43ca8f48bd7e48b4a24f980194cc7c89', 'request_id': 'req-5f1079f4-a9a6-4984-931b-1e764b2c98c6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:33.552190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5f1079f4-a9a6-4984-931b-1e764b2c98c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43ca8f48bd7e48b4a24f980194cc7c89', 'request_id': 'req-5f1079f4-a9a6-4984-931b-1e764b2c98c6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:33.581216 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5f1079f4-a9a6-4984-931b-1e764b2c98c6 admin admin] The token cache is being invalidate because project 43ca8f48bd7e48b4a24f980194cc7c89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:33.581772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5f1079f4-a9a6-4984-931b-1e764b2c98c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:33.582966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5f1079f4-a9a6-4984-931b-1e764b2c98c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:33.584904 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 971/1931] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:33 2022] DELETE /identity/v3/projects/43ca8f48bd7e48b4a24f980194cc7c89 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:33.588578 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a6f13-8a74-44c0-82c2-269cbc4e76b0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:33.589849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a6f13-8a74-44c0-82c2-269cbc4e76b0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:33.590350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0a6f13-8a74-44c0-82c2-269cbc4e76b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:33.618476 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3a0a6f13-8a74-44c0-82c2-269cbc4e76b0 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:33.618723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a3675b-bf28-430a-bd14-6036d0b30865 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:33.619712 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3a0a6f13-8a74-44c0-82c2-269cbc4e76b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:33.623590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b053d-0854-4858-b967-e0c4fef4d800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:33.624261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b053d-0854-4858-b967-e0c4fef4d800 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:33.624539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b053d-0854-4858-b967-e0c4fef4d800 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:33.624824 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83b053d-0854-4858-b967-e0c4fef4d800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:33.626872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a83b053d-0854-4858-b967-e0c4fef4d800 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:33.717032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 972/1932] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:33.717032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43469d3c-5a96-401a-bd06-204551de6a68 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:33.717032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43469d3c-5a96-401a-bd06-204551de6a68 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:33.717032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43469d3c-5a96-401a-bd06-204551de6a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:33.725951 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b053d-0854-4858-b967-e0c4fef4d800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:33.727562 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83b053d-0854-4858-b967-e0c4fef4d800 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:33.737705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 961/1933] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:33 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:33.738522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b053d-0854-4858-b967-e0c4fef4d800 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:33.739860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a83b053d-0854-4858-b967-e0c4fef4d800 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:33.741802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-43469d3c-5a96-401a-bd06-204551de6a68 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:33.742984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-43469d3c-5a96-401a-bd06-204551de6a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:33.751213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 973/1934] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:33.759533 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c446f94-d3bf-40c6-be3b-9960385cecac None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:33.760363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c446f94-d3bf-40c6-be3b-9960385cecac None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:33.760899 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c446f94-d3bf-40c6-be3b-9960385cecac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:33.791349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0c446f94-d3bf-40c6-be3b-9960385cecac None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:33.792913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0c446f94-d3bf-40c6-be3b-9960385cecac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:33.807069 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 974/1935] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:14:33.811410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c446f94-d3bf-40c6-be3b-9960385cecac None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:33.815055 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:33.820125 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83b053d-0854-4858-b967-e0c4fef4d800 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:33.827074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b595b-fb86-4c69-bddf-0be93a786dde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:33.828228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b595b-fb86-4c69-bddf-0be93a786dde service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:33.828694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b595b-fb86-4c69-bddf-0be93a786dde service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:33.829121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64b595b-fb86-4c69-bddf-0be93a786dde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:33.838383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c446f94-d3bf-40c6-be3b-9960385cecac None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:33.840897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b595b-fb86-4c69-bddf-0be93a786dde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:33.842704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64b595b-fb86-4c69-bddf-0be93a786dde service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:33.844893 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f8367-61c9-4078-88ba-4e70a3f03497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:33.851832 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f8367-61c9-4078-88ba-4e70a3f03497 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:33.852163 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f8367-61c9-4078-88ba-4e70a3f03497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:33.852441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183f8367-61c9-4078-88ba-4e70a3f03497 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:33.853240 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f8367-61c9-4078-88ba-4e70a3f03497 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:14:33.853551 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f8367-61c9-4078-88ba-4e70a3f03497 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:33.856397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 962/1936] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:33.857017 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183f8367-61c9-4078-88ba-4e70a3f03497 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:33.872970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 975/1937] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:14:33.875901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b595b-fb86-4c69-bddf-0be93a786dde service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:33.888319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64b595b-fb86-4c69-bddf-0be93a786dde service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:33.894023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e5682-809e-4906-8a95-b422996ac829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:33.894793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e5682-809e-4906-8a95-b422996ac829 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:33.895279 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e5682-809e-4906-8a95-b422996ac829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:33.896049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e5682-809e-4906-8a95-b422996ac829 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:33.897677 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e5682-809e-4906-8a95-b422996ac829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:33.899891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e5682-809e-4906-8a95-b422996ac829 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:33.907386 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f8367-61c9-4078-88ba-4e70a3f03497 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:33.910184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-183f8367-61c9-4078-88ba-4e70a3f03497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:33.927536 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183f8367-61c9-4078-88ba-4e70a3f03497 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:33.935698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2daca-5232-40f8-9225-be74cad02589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:33.936524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2daca-5232-40f8-9225-be74cad02589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:33.936853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2daca-5232-40f8-9225-be74cad02589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:33.937161 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2daca-5232-40f8-9225-be74cad02589 admin admin] PATH_INFO: `/v3/users/2536887cef034f189be889a0e9c35a16` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:33.939154 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2daca-5232-40f8-9225-be74cad02589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2536887cef034f189be889a0e9c35a16)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:33.939334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 963/1938] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:33 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:14:33.942172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2daca-5232-40f8-9225-be74cad02589 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:33.946843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e5682-809e-4906-8a95-b422996ac829 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:34.006018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-38b2daca-5232-40f8-9225-be74cad02589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2536887cef034f189be889a0e9c35a16'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:34.007262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-38b2daca-5232-40f8-9225-be74cad02589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2536887cef034f189be889a0e9c35a16'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:34.020666 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-38b2daca-5232-40f8-9225-be74cad02589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2536887cef034f189be889a0e9c35a16'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:34.026069 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e5682-809e-4906-8a95-b422996ac829 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:34.029271 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-38b2daca-5232-40f8-9225-be74cad02589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2536887cef034f189be889a0e9c35a16'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:34.034838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c23732-31b5-475f-bd55-7e765c6d0eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:34.035590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c23732-31b5-475f-bd55-7e765c6d0eda admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:34.035881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c23732-31b5-475f-bd55-7e765c6d0eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:34.036161 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c23732-31b5-475f-bd55-7e765c6d0eda admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:34.036877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c23732-31b5-475f-bd55-7e765c6d0eda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:34.041005 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c23732-31b5-475f-bd55-7e765c6d0eda admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:34.047528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-38b2daca-5232-40f8-9225-be74cad02589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2536887cef034f189be889a0e9c35a16'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:34.054973 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 976/1939] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:33 2022] DELETE /identity/v3/users/2536887cef034f189be889a0e9c35a16 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:34.060382 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e8ba8-a239-4850-b026-16d7e2019d39 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:34.060803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e8ba8-a239-4850-b026-16d7e2019d39 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:34.061201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e8ba8-a239-4850-b026-16d7e2019d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:34.069585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 964/1940] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:33 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:34.071236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c23732-31b5-475f-bd55-7e765c6d0eda admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:34.073117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-56c23732-31b5-475f-bd55-7e765c6d0eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:34.085327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-365e8ba8-a239-4850-b026-16d7e2019d39 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:34.085956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c23732-31b5-475f-bd55-7e765c6d0eda admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:34.086768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-365e8ba8-a239-4850-b026-16d7e2019d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:34.102283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:34.159114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b059be7-8500-4569-8d42-7ef864eb008e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:34.159798 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b059be7-8500-4569-8d42-7ef864eb008e admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:34.160494 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b059be7-8500-4569-8d42-7ef864eb008e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:34.160772 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b059be7-8500-4569-8d42-7ef864eb008e admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:34.161980 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b059be7-8500-4569-8d42-7ef864eb008e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:34.163788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b059be7-8500-4569-8d42-7ef864eb008e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:34.176085 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 965/1941] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:34.178414 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 977/1942] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:14:34.181413 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b059be7-8500-4569-8d42-7ef864eb008e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:34.189297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e8ba8-a239-4850-b026-16d7e2019d39 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:34.190669 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-365e8ba8-a239-4850-b026-16d7e2019d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:34.212854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b059be7-8500-4569-8d42-7ef864eb008e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:34.217837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91892644-8830-4c21-96ce-fda5b13a7fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:34.218580 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91892644-8830-4c21-96ce-fda5b13a7fad admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:34.218870 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91892644-8830-4c21-96ce-fda5b13a7fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:34.219149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91892644-8830-4c21-96ce-fda5b13a7fad admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:34.219883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91892644-8830-4c21-96ce-fda5b13a7fad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:34.221551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91892644-8830-4c21-96ce-fda5b13a7fad admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:34.232378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 966/1943] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:34 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:34.235328 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91892644-8830-4c21-96ce-fda5b13a7fad admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:34.238092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-91892644-8830-4c21-96ce-fda5b13a7fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:34.256101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e8ba8-a239-4850-b026-16d7e2019d39 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:34.260666 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17e71a-c954-464a-8f20-3eaf05957f69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:34.260976 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91892644-8830-4c21-96ce-fda5b13a7fad admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:34.261355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17e71a-c954-464a-8f20-3eaf05957f69 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:34.261641 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17e71a-c954-464a-8f20-3eaf05957f69 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:34.262355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db17e71a-c954-464a-8f20-3eaf05957f69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:34.267715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc76186-1334-4b31-91ac-460d4f4c2c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:34.268721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc76186-1334-4b31-91ac-460d4f4c2c86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:34.269083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc76186-1334-4b31-91ac-460d4f4c2c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:34.269507 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc76186-1334-4b31-91ac-460d4f4c2c86 admin admin] PATH_INFO: `/v3/projects/8c062d01b1054cd498c113d2e3894d56` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:34.270816 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc76186-1334-4b31-91ac-460d4f4c2c86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c062d01b1054cd498c113d2e3894d56)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:34.273435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc76186-1334-4b31-91ac-460d4f4c2c86 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:34.275149 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17e71a-c954-464a-8f20-3eaf05957f69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:34.277018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db17e71a-c954-464a-8f20-3eaf05957f69 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:34.301253 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 978/1944] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:14:34.302427 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17e71a-c954-464a-8f20-3eaf05957f69 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:34.329401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db17e71a-c954-464a-8f20-3eaf05957f69 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:34.339244 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fef6e4-dc13-4394-8874-5e702cd6f86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:34.340798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fef6e4-dc13-4394-8874-5e702cd6f86b admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:34.341110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fef6e4-dc13-4394-8874-5e702cd6f86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:34.341389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fef6e4-dc13-4394-8874-5e702cd6f86b admin admin] PATH_INFO: `/v3/projects/c1ca205e8be148c8b724b1603111ce65/users/1e2e475aa45440faaa336870675cf9e9/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:34.345283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:34.364919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5dc76186-1334-4b31-91ac-460d4f4c2c86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c062d01b1054cd498c113d2e3894d56', 'request_id': 'req-5dc76186-1334-4b31-91ac-460d4f4c2c86'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:34.376191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fef6e4-dc13-4394-8874-5e702cd6f86b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1ca205e8be148c8b724b1603111ce65, user_id=1e2e475aa45440faaa336870675cf9e9, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:34.380308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fef6e4-dc13-4394-8874-5e702cd6f86b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:34.382203 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5dc76186-1334-4b31-91ac-460d4f4c2c86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c062d01b1054cd498c113d2e3894d56', 'request_id': 'req-5dc76186-1334-4b31-91ac-460d4f4c2c86'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:34.398234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5dc76186-1334-4b31-91ac-460d4f4c2c86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c062d01b1054cd498c113d2e3894d56', 'request_id': 'req-5dc76186-1334-4b31-91ac-460d4f4c2c86'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:34.411504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 979/1945] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:34 2022] PUT /identity/v3/projects/c1ca205e8be148c8b724b1603111ce65/users/1e2e475aa45440faaa336870675cf9e9/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:34.420154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5dc76186-1334-4b31-91ac-460d4f4c2c86 admin admin] The token cache is being invalidate because project 8c062d01b1054cd498c113d2e3894d56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:34.420154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5dc76186-1334-4b31-91ac-460d4f4c2c86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:34.420154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5dc76186-1334-4b31-91ac-460d4f4c2c86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:34.420154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 967/1946] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:34 2022] DELETE /identity/v3/projects/8c062d01b1054cd498c113d2e3894d56 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:34.421816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a3db7-b7d3-4a4a-898e-c465d36be359 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:34.422107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a3db7-b7d3-4a4a-898e-c465d36be359 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:34.422383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a3db7-b7d3-4a4a-898e-c465d36be359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:34.425716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc76186-1334-4b31-91ac-460d4f4c2c86 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:34.427152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5dc76186-1334-4b31-91ac-460d4f4c2c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:34.480562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5e4a3db7-b7d3-4a4a-898e-c465d36be359 None None] MFA Rules not processed for user `1e2e475aa45440faaa336870675cf9e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:34.483707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5e4a3db7-b7d3-4a4a-898e-c465d36be359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:34.527152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc76186-1334-4b31-91ac-460d4f4c2c86 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:34.609053 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 980/1947] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:34.620443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a3db7-b7d3-4a4a-898e-c465d36be359 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:34.622597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5e4a3db7-b7d3-4a4a-898e-c465d36be359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:34.632280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ac955-4729-4867-ae10-414281d695cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:34.633696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ac955-4729-4867-ae10-414281d695cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:34.635230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ac955-4729-4867-ae10-414281d695cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:34.635731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9ac955-4729-4867-ae10-414281d695cb admin admin] PATH_INFO: `/v3/users/2abfac1ab96e46678da133cf7ffe9cd6` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:34.637872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ac955-4729-4867-ae10-414281d695cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=2abfac1ab96e46678da133cf7ffe9cd6)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:34.640150 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9ac955-4729-4867-ae10-414281d695cb admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:34.668273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:34.716020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ea9ac955-4729-4867-ae10-414281d695cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2abfac1ab96e46678da133cf7ffe9cd6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:34.716293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a3db7-b7d3-4a4a-898e-c465d36be359 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:34.717492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ea9ac955-4729-4867-ae10-414281d695cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2abfac1ab96e46678da133cf7ffe9cd6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:34.729396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43036130-0a3f-4716-822b-cbea4da42aa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:34.731065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43036130-0a3f-4716-822b-cbea4da42aa1 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:34.731369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43036130-0a3f-4716-822b-cbea4da42aa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:34.731638 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43036130-0a3f-4716-822b-cbea4da42aa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:34.733767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-43036130-0a3f-4716-822b-cbea4da42aa1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:34.735195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ea9ac955-4729-4867-ae10-414281d695cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2abfac1ab96e46678da133cf7ffe9cd6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:34.745741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ea9ac955-4729-4867-ae10-414281d695cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2abfac1ab96e46678da133cf7ffe9cd6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:34.748790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:34.750449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43036130-0a3f-4716-822b-cbea4da42aa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:34.754258 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43036130-0a3f-4716-822b-cbea4da42aa1 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:34.764042 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ea9ac955-4729-4867-ae10-414281d695cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2abfac1ab96e46678da133cf7ffe9cd6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:34.772375 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 968/1948] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:34 2022] DELETE /identity/v3/users/2abfac1ab96e46678da133cf7ffe9cd6 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:34.796538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 981/1949] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:35.133670 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ac955-4729-4867-ae10-414281d695cb admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:35.135142 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ea9ac955-4729-4867-ae10-414281d695cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:35.318728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9ac955-4729-4867-ae10-414281d695cb admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:35.341917 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b12a6-1979-4d53-96ee-e8c893b7e82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:35.342812 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b12a6-1979-4d53-96ee-e8c893b7e82f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:35.343096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b12a6-1979-4d53-96ee-e8c893b7e82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:35.343384 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07b12a6-1979-4d53-96ee-e8c893b7e82f admin admin] PATH_INFO: `/v3/projects/7d38092cafa94ce2838aef4ae7968c1f` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:35.347010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b12a6-1979-4d53-96ee-e8c893b7e82f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d38092cafa94ce2838aef4ae7968c1f)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:35.349151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07b12a6-1979-4d53-96ee-e8c893b7e82f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:35.484297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e07b12a6-1979-4d53-96ee-e8c893b7e82f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d38092cafa94ce2838aef4ae7968c1f', 'request_id': 'req-e07b12a6-1979-4d53-96ee-e8c893b7e82f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:35.490538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e07b12a6-1979-4d53-96ee-e8c893b7e82f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d38092cafa94ce2838aef4ae7968c1f', 'request_id': 'req-e07b12a6-1979-4d53-96ee-e8c893b7e82f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:35.498410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e07b12a6-1979-4d53-96ee-e8c893b7e82f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d38092cafa94ce2838aef4ae7968c1f', 'request_id': 'req-e07b12a6-1979-4d53-96ee-e8c893b7e82f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:35.510394 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e07b12a6-1979-4d53-96ee-e8c893b7e82f admin admin] The token cache is being invalidate because project 7d38092cafa94ce2838aef4ae7968c1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:35.510696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e07b12a6-1979-4d53-96ee-e8c893b7e82f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:35.511349 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e07b12a6-1979-4d53-96ee-e8c893b7e82f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:35.513588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 969/1950] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:35 2022] DELETE /identity/v3/projects/7d38092cafa94ce2838aef4ae7968c1f => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:35.520571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b12a6-1979-4d53-96ee-e8c893b7e82f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:35.521480 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43036130-0a3f-4716-822b-cbea4da42aa1 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:35.523381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e07b12a6-1979-4d53-96ee-e8c893b7e82f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:35.537062 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43036130-0a3f-4716-822b-cbea4da42aa1 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:35.595101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07b12a6-1979-4d53-96ee-e8c893b7e82f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:35.599685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae35f3-930b-47c1-a9e2-f391002013e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:35.600655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae35f3-930b-47c1-a9e2-f391002013e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:35.601064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae35f3-930b-47c1-a9e2-f391002013e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:35.601339 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae35f3-930b-47c1-a9e2-f391002013e5 admin admin] PATH_INFO: `/v3/projects/75a351922ccf4414b873a03ca85ba84b` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:35.602648 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae35f3-930b-47c1-a9e2-f391002013e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75a351922ccf4414b873a03ca85ba84b)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:35.604675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae35f3-930b-47c1-a9e2-f391002013e5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:35.606307 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce415be-402f-4c0a-a652-90829985196d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:35.607199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce415be-402f-4c0a-a652-90829985196d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:35.607463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce415be-402f-4c0a-a652-90829985196d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:35.607730 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce415be-402f-4c0a-a652-90829985196d admin admin] PATH_INFO: `/v3/users/8b33138c7f42495787adb6eb8f87ef24` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:35.608988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce415be-402f-4c0a-a652-90829985196d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b33138c7f42495787adb6eb8f87ef24)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:35.610919 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce415be-402f-4c0a-a652-90829985196d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:35.638075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:35.658030 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d1ae35f3-930b-47c1-a9e2-f391002013e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75a351922ccf4414b873a03ca85ba84b', 'request_id': 'req-d1ae35f3-930b-47c1-a9e2-f391002013e5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:35.661640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ce415be-402f-4c0a-a652-90829985196d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b33138c7f42495787adb6eb8f87ef24'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:35.662296 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ce415be-402f-4c0a-a652-90829985196d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b33138c7f42495787adb6eb8f87ef24'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:35.665008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d1ae35f3-930b-47c1-a9e2-f391002013e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75a351922ccf4414b873a03ca85ba84b', 'request_id': 'req-d1ae35f3-930b-47c1-a9e2-f391002013e5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:35.672899 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d1ae35f3-930b-47c1-a9e2-f391002013e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75a351922ccf4414b873a03ca85ba84b', 'request_id': 'req-d1ae35f3-930b-47c1-a9e2-f391002013e5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:35.676055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ce415be-402f-4c0a-a652-90829985196d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b33138c7f42495787adb6eb8f87ef24'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:35.683307 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ce415be-402f-4c0a-a652-90829985196d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b33138c7f42495787adb6eb8f87ef24'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:35.684657 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d1ae35f3-930b-47c1-a9e2-f391002013e5 admin admin] The token cache is being invalidate because project 75a351922ccf4414b873a03ca85ba84b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:35.684970 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d1ae35f3-930b-47c1-a9e2-f391002013e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:35.685621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d1ae35f3-930b-47c1-a9e2-f391002013e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:35.687185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 970/1951] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:35 2022] DELETE /identity/v3/projects/75a351922ccf4414b873a03ca85ba84b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:35.691031 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ce415be-402f-4c0a-a652-90829985196d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b33138c7f42495787adb6eb8f87ef24'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:35.694666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85262f-c432-4ecc-a623-7a03bd021be9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:35.695050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85262f-c432-4ecc-a623-7a03bd021be9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:35.695470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed85262f-c432-4ecc-a623-7a03bd021be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:35.701529 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 982/1952] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:35 2022] DELETE /identity/v3/users/8b33138c7f42495787adb6eb8f87ef24 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:35.720868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ed85262f-c432-4ecc-a623-7a03bd021be9 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:35.722521 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ed85262f-c432-4ecc-a623-7a03bd021be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:35.794727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 971/1953] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:14:35.803424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68906a94-2d90-4129-af5d-b2b4686e2f3b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:35.803963 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68906a94-2d90-4129-af5d-b2b4686e2f3b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:35.803963 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68906a94-2d90-4129-af5d-b2b4686e2f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:35.828876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-68906a94-2d90-4129-af5d-b2b4686e2f3b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:35.830305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-68906a94-2d90-4129-af5d-b2b4686e2f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:35.841096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 983/1954] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:35.845679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85262f-c432-4ecc-a623-7a03bd021be9 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:35.856610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed85262f-c432-4ecc-a623-7a03bd021be9 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:35.861449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68770cf0-88db-4250-92d0-14ae6ae4e87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:35.862136 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68770cf0-88db-4250-92d0-14ae6ae4e87e admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:35.862419 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68770cf0-88db-4250-92d0-14ae6ae4e87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:35.862694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68770cf0-88db-4250-92d0-14ae6ae4e87e admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:35.863463 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68770cf0-88db-4250-92d0-14ae6ae4e87e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:14:35.863770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68770cf0-88db-4250-92d0-14ae6ae4e87e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:35.865441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68770cf0-88db-4250-92d0-14ae6ae4e87e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:35.880962 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 972/1955] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:14:35.883900 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68906a94-2d90-4129-af5d-b2b4686e2f3b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:35.893452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68906a94-2d90-4129-af5d-b2b4686e2f3b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:35.898631 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435670a3-4b39-49a2-9fc8-ca414ae207ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:35.899291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435670a3-4b39-49a2-9fc8-ca414ae207ef admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:35.899569 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435670a3-4b39-49a2-9fc8-ca414ae207ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:35.899843 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435670a3-4b39-49a2-9fc8-ca414ae207ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:35.900534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435670a3-4b39-49a2-9fc8-ca414ae207ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:35.902738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435670a3-4b39-49a2-9fc8-ca414ae207ef admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:35.930978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 984/1956] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:35 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:14:35.934896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68770cf0-88db-4250-92d0-14ae6ae4e87e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:35.945974 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68770cf0-88db-4250-92d0-14ae6ae4e87e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:36.008976 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a022030-8504-46b6-88d3-303c6b364039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:36.009603 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a022030-8504-46b6-88d3-303c6b364039 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:36.009887 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a022030-8504-46b6-88d3-303c6b364039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:36.010163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a022030-8504-46b6-88d3-303c6b364039 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:36.010873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a022030-8504-46b6-88d3-303c6b364039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:36.012793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a022030-8504-46b6-88d3-303c6b364039 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:36.050629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 973/1957] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:35 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:36.051887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435670a3-4b39-49a2-9fc8-ca414ae207ef admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:36.067473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435670a3-4b39-49a2-9fc8-ca414ae207ef admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:36.074223 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60e240-a80e-4cb6-8b37-f4efc37b7d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:36.075035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60e240-a80e-4cb6-8b37-f4efc37b7d92 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:36.075433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60e240-a80e-4cb6-8b37-f4efc37b7d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:36.075801 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60e240-a80e-4cb6-8b37-f4efc37b7d92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:36.076610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60e240-a80e-4cb6-8b37-f4efc37b7d92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:36.078665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60e240-a80e-4cb6-8b37-f4efc37b7d92 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:36.096556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 985/1958] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:36.097666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a022030-8504-46b6-88d3-303c6b364039 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:36.109425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a022030-8504-46b6-88d3-303c6b364039 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:36.130634 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1438070-d15d-4b6a-b66b-16acbea40ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:36.132071 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1438070-d15d-4b6a-b66b-16acbea40ed3 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:36.132442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1438070-d15d-4b6a-b66b-16acbea40ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:36.132771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1438070-d15d-4b6a-b66b-16acbea40ed3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:36.133919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1438070-d15d-4b6a-b66b-16acbea40ed3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:36.135660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1438070-d15d-4b6a-b66b-16acbea40ed3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:36.158812 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60e240-a80e-4cb6-8b37-f4efc37b7d92 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:36.159008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 974/1959] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:36.193246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60e240-a80e-4cb6-8b37-f4efc37b7d92 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:36.201197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299955e5-a81d-4ba4-99da-0e95946fb73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:36.202068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299955e5-a81d-4ba4-99da-0e95946fb73d admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:36.202680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299955e5-a81d-4ba4-99da-0e95946fb73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:36.202680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299955e5-a81d-4ba4-99da-0e95946fb73d admin admin] PATH_INFO: `/v3/projects/49d652469bad4f92ad11f2fb3b50a00c/users/68923614145943ba85d77d9d4a84ca72/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:36.210582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:36.232211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299955e5-a81d-4ba4-99da-0e95946fb73d admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d652469bad4f92ad11f2fb3b50a00c, user_id=68923614145943ba85d77d9d4a84ca72, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:36.239462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299955e5-a81d-4ba4-99da-0e95946fb73d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:36.266731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 986/1960] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:36 2022] PUT /identity/v3/projects/49d652469bad4f92ad11f2fb3b50a00c/users/68923614145943ba85d77d9d4a84ca72/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:36.268555 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d1524-3cbc-4082-99b1-162b72fdc684 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:36.268878 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d1524-3cbc-4082-99b1-162b72fdc684 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:36.269221 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d1524-3cbc-4082-99b1-162b72fdc684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:36.305500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-201d1524-3cbc-4082-99b1-162b72fdc684 None None] MFA Rules not processed for user `68923614145943ba85d77d9d4a84ca72`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:36.306845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-201d1524-3cbc-4082-99b1-162b72fdc684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:36.315629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:36.420094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 975/1961] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:36.434945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299955e5-a81d-4ba4-99da-0e95946fb73d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:36.443761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-299955e5-a81d-4ba4-99da-0e95946fb73d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:36.563089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d1524-3cbc-4082-99b1-162b72fdc684 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:36.563545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299955e5-a81d-4ba4-99da-0e95946fb73d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:36.564957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-201d1524-3cbc-4082-99b1-162b72fdc684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:36.568188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a6b38-a119-426c-b562-5856354334b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:36.569234 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a6b38-a119-426c-b562-5856354334b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:36.570276 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a6b38-a119-426c-b562-5856354334b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:36.570674 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a6b38-a119-426c-b562-5856354334b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:36.583742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a6b38-a119-426c-b562-5856354334b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:36.585756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a6b38-a119-426c-b562-5856354334b6 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:36.599931 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 987/1962] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:36.719259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d1524-3cbc-4082-99b1-162b72fdc684 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:36.744601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d751e6be-2e03-4817-8ccf-b6318d0e4ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:36.744601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751e6be-2e03-4817-8ccf-b6318d0e4ec7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:36.744601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751e6be-2e03-4817-8ccf-b6318d0e4ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:36.744601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d751e6be-2e03-4817-8ccf-b6318d0e4ec7 admin admin] PATH_INFO: `/v3/projects/ca443abe0e44403086a7dab7ef3f379a` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:36.745266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751e6be-2e03-4817-8ccf-b6318d0e4ec7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca443abe0e44403086a7dab7ef3f379a)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:36.745266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d751e6be-2e03-4817-8ccf-b6318d0e4ec7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:36.816304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d751e6be-2e03-4817-8ccf-b6318d0e4ec7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca443abe0e44403086a7dab7ef3f379a', 'request_id': 'req-d751e6be-2e03-4817-8ccf-b6318d0e4ec7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:36.824199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d751e6be-2e03-4817-8ccf-b6318d0e4ec7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca443abe0e44403086a7dab7ef3f379a', 'request_id': 'req-d751e6be-2e03-4817-8ccf-b6318d0e4ec7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:36.838875 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d751e6be-2e03-4817-8ccf-b6318d0e4ec7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca443abe0e44403086a7dab7ef3f379a', 'request_id': 'req-d751e6be-2e03-4817-8ccf-b6318d0e4ec7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:36.864131 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d751e6be-2e03-4817-8ccf-b6318d0e4ec7 admin admin] The token cache is being invalidate because project ca443abe0e44403086a7dab7ef3f379a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:36.864433 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d751e6be-2e03-4817-8ccf-b6318d0e4ec7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:36.865228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d751e6be-2e03-4817-8ccf-b6318d0e4ec7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:36.871468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 976/1963] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:36 2022] DELETE /identity/v3/projects/ca443abe0e44403086a7dab7ef3f379a => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:14:36.874091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb827bb-5c9e-47b4-a83a-b31438c81f1d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:36.874572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb827bb-5c9e-47b4-a83a-b31438c81f1d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:36.874749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb827bb-5c9e-47b4-a83a-b31438c81f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:36.903715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-dbb827bb-5c9e-47b4-a83a-b31438c81f1d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:36.904950 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-dbb827bb-5c9e-47b4-a83a-b31438c81f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:37.004669 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 988/1964] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:14:37.012977 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf7b3f-5ad1-4b79-b8aa-173946b4f70f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:37.013247 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf7b3f-5ad1-4b79-b8aa-173946b4f70f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:37.013528 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cf7b3f-5ad1-4b79-b8aa-173946b4f70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:37.047185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-37cf7b3f-5ad1-4b79-b8aa-173946b4f70f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:37.051252 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-37cf7b3f-5ad1-4b79-b8aa-173946b4f70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:37.064581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 977/1965] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:37.067665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb827bb-5c9e-47b4-a83a-b31438c81f1d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:37.082708 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb827bb-5c9e-47b4-a83a-b31438c81f1d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:37.089658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5137be9-65d7-43f2-8742-6783327fdaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:37.090824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5137be9-65d7-43f2-8742-6783327fdaad admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:37.091277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5137be9-65d7-43f2-8742-6783327fdaad admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:37.091746 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5137be9-65d7-43f2-8742-6783327fdaad admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:37.093011 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5137be9-65d7-43f2-8742-6783327fdaad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:14:37.094359 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5137be9-65d7-43f2-8742-6783327fdaad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:37.097724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5137be9-65d7-43f2-8742-6783327fdaad admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:37.118588 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 989/1966] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:14:37.120512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf7b3f-5ad1-4b79-b8aa-173946b4f70f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:37.154074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cf7b3f-5ad1-4b79-b8aa-173946b4f70f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:37.164064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfbdb3-a851-4ac4-90ce-bbd422f2e1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:37.164726 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfbdb3-a851-4ac4-90ce-bbd422f2e1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:37.165077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfbdb3-a851-4ac4-90ce-bbd422f2e1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:37.165325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebfbdb3-a851-4ac4-90ce-bbd422f2e1d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:37.166093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebfbdb3-a851-4ac4-90ce-bbd422f2e1d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:37.167964 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebfbdb3-a851-4ac4-90ce-bbd422f2e1d5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:37.245446 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 978/1967] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:37 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:37.249448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5137be9-65d7-43f2-8742-6783327fdaad admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:37.265320 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5137be9-65d7-43f2-8742-6783327fdaad admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:37.355243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fcc84-464e-4131-8aa1-f4f6b43341b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:37.355987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fcc84-464e-4131-8aa1-f4f6b43341b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:37.356271 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fcc84-464e-4131-8aa1-f4f6b43341b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:37.357105 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913fcc84-464e-4131-8aa1-f4f6b43341b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:37.357266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fcc84-464e-4131-8aa1-f4f6b43341b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:37.359151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913fcc84-464e-4131-8aa1-f4f6b43341b8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:37.381906 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 990/1968] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:37 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:37.386048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfbdb3-a851-4ac4-90ce-bbd422f2e1d5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:37.399247 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebfbdb3-a851-4ac4-90ce-bbd422f2e1d5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:37.403149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe63cf1-a911-4963-9188-5c3e5243d48e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:37.404101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe63cf1-a911-4963-9188-5c3e5243d48e admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:37.404378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe63cf1-a911-4963-9188-5c3e5243d48e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:37.404648 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe63cf1-a911-4963-9188-5c3e5243d48e admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:37.405423 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe63cf1-a911-4963-9188-5c3e5243d48e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:37.407014 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe63cf1-a911-4963-9188-5c3e5243d48e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:37.412633 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:37.418738 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 979/1969] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:37.422985 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fcc84-464e-4131-8aa1-f4f6b43341b8 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:37.428120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:37.434627 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913fcc84-464e-4131-8aa1-f4f6b43341b8 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:37.439862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfc07c5-1c59-4cd9-9d99-c209c3ef3d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:37.440758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc07c5-1c59-4cd9-9d99-c209c3ef3d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:37.441067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc07c5-1c59-4cd9-9d99-c209c3ef3d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:37.441340 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfc07c5-1c59-4cd9-9d99-c209c3ef3d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:37.442097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc07c5-1c59-4cd9-9d99-c209c3ef3d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:37.443674 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfc07c5-1c59-4cd9-9d99-c209c3ef3d14 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:37.457497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 991/1970] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:37.458956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe63cf1-a911-4963-9188-5c3e5243d48e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:37.469470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe63cf1-a911-4963-9188-5c3e5243d48e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:37.474240 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008f082-6837-4f70-9510-012148445074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:37.475245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008f082-6837-4f70-9510-012148445074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:37.475459 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008f082-6837-4f70-9510-012148445074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:37.475694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008f082-6837-4f70-9510-012148445074 admin admin] PATH_INFO: `/v3/projects/31472a4e05bb45b487df47f4f8255789/users/a8376dde1ab64dbdb1f6c37de2ee32be/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:37.491840 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008f082-6837-4f70-9510-012148445074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31472a4e05bb45b487df47f4f8255789, user_id=a8376dde1ab64dbdb1f6c37de2ee32be, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:37.502327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008f082-6837-4f70-9510-012148445074 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:37.508341 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 980/1971] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:37 2022] PUT /identity/v3/projects/31472a4e05bb45b487df47f4f8255789/users/a8376dde1ab64dbdb1f6c37de2ee32be/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:37.518547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf21f3-311d-4fce-98de-67dfec3a582d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:37.518903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf21f3-311d-4fce-98de-67dfec3a582d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:37.519182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf21f3-311d-4fce-98de-67dfec3a582d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:37.545720 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-32cf21f3-311d-4fce-98de-67dfec3a582d None None] MFA Rules not processed for user `a8376dde1ab64dbdb1f6c37de2ee32be`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:37.546835 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-32cf21f3-311d-4fce-98de-67dfec3a582d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:37.609490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 992/1972] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:37.613933 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008f082-6837-4f70-9510-012148445074 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:37.675032 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008f082-6837-4f70-9510-012148445074 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:37.680237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd91c2-2ef7-4860-86a0-27a2ec5dc32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:37.680891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd91c2-2ef7-4860-86a0-27a2ec5dc32e admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:37.681206 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd91c2-2ef7-4860-86a0-27a2ec5dc32e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:37.681483 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bd91c2-2ef7-4860-86a0-27a2ec5dc32e admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:37.682183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd91c2-2ef7-4860-86a0-27a2ec5dc32e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:37.684083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bd91c2-2ef7-4860-86a0-27a2ec5dc32e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:37.711173 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 981/1973] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:37 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:37.714619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf21f3-311d-4fce-98de-67dfec3a582d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:37.724936 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf21f3-311d-4fce-98de-67dfec3a582d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:37.780730 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f64218-5929-4f2f-82ef-1e255e7021a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:37.781437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f64218-5929-4f2f-82ef-1e255e7021a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:37.781718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f64218-5929-4f2f-82ef-1e255e7021a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:37.781995 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f64218-5929-4f2f-82ef-1e255e7021a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:37.782689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f64218-5929-4f2f-82ef-1e255e7021a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:37.784586 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f64218-5929-4f2f-82ef-1e255e7021a3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:37.804744 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 993/1974] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:37 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:14:37.808809 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd91c2-2ef7-4860-86a0-27a2ec5dc32e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:37.818896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bd91c2-2ef7-4860-86a0-27a2ec5dc32e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:37.823498 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf00bb-abc2-4095-a68c-b514bc2898e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:37.824152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf00bb-abc2-4095-a68c-b514bc2898e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:37.824437 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf00bb-abc2-4095-a68c-b514bc2898e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:37.824716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf00bb-abc2-4095-a68c-b514bc2898e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:37.825484 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf00bb-abc2-4095-a68c-b514bc2898e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:37.827070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf00bb-abc2-4095-a68c-b514bc2898e1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:37.837409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 982/1975] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:14:37.841395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f64218-5929-4f2f-82ef-1e255e7021a3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:37.851718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f64218-5929-4f2f-82ef-1e255e7021a3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:37.856244 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b852e82-cee5-40d5-8758-9759b073a09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:37.857156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b852e82-cee5-40d5-8758-9759b073a09b admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:37.857437 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b852e82-cee5-40d5-8758-9759b073a09b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:37.857711 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b852e82-cee5-40d5-8758-9759b073a09b admin admin] PATH_INFO: `/v3/projects/e1d339ff2a7742a0be32c4c4e2c12090/users/4915cba7c6c644d5ae26615a66f326ff/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:37.872788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b852e82-cee5-40d5-8758-9759b073a09b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d339ff2a7742a0be32c4c4e2c12090, user_id=4915cba7c6c644d5ae26615a66f326ff, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:37.875518 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b852e82-cee5-40d5-8758-9759b073a09b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:37.888115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 994/1976] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:37 2022] PUT /identity/v3/projects/e1d339ff2a7742a0be32c4c4e2c12090/users/4915cba7c6c644d5ae26615a66f326ff/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:37.892010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf00bb-abc2-4095-a68c-b514bc2898e1 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:37.901982 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf00bb-abc2-4095-a68c-b514bc2898e1 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:37.957856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615dfb26-3a51-4ec4-a518-f4015b6cba81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:37.958514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615dfb26-3a51-4ec4-a518-f4015b6cba81 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:37.958797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615dfb26-3a51-4ec4-a518-f4015b6cba81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:37.959078 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615dfb26-3a51-4ec4-a518-f4015b6cba81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:37.959794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615dfb26-3a51-4ec4-a518-f4015b6cba81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:37.961409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615dfb26-3a51-4ec4-a518-f4015b6cba81 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:37.970929 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 983/1977] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:37.975641 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b852e82-cee5-40d5-8758-9759b073a09b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:37.986001 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b852e82-cee5-40d5-8758-9759b073a09b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:37.990765 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2702d84-02c8-4f04-8917-69b5a02a9f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:37.992162 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2702d84-02c8-4f04-8917-69b5a02a9f35 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:37.992442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2702d84-02c8-4f04-8917-69b5a02a9f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:37.992718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2702d84-02c8-4f04-8917-69b5a02a9f35 admin admin] PATH_INFO: `/v3/projects/e1d339ff2a7742a0be32c4c4e2c12090/users/4915cba7c6c644d5ae26615a66f326ff/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:37.994787 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2702d84-02c8-4f04-8917-69b5a02a9f35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d339ff2a7742a0be32c4c4e2c12090, user_id=4915cba7c6c644d5ae26615a66f326ff, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:37.997465 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2702d84-02c8-4f04-8917-69b5a02a9f35 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:38.008108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 995/1978] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:37 2022] PUT /identity/v3/projects/e1d339ff2a7742a0be32c4c4e2c12090/users/4915cba7c6c644d5ae26615a66f326ff/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:14:38.019254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714761b-450a-4fde-aac7-9882648c0151 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:38.019539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714761b-450a-4fde-aac7-9882648c0151 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:38.019813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714761b-450a-4fde-aac7-9882648c0151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:38.047183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7714761b-450a-4fde-aac7-9882648c0151 None None] MFA Rules not processed for user `4915cba7c6c644d5ae26615a66f326ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:38.048688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7714761b-450a-4fde-aac7-9882648c0151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:38.084902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:38.120837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 984/1979] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:38.132307 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2702d84-02c8-4f04-8917-69b5a02a9f35 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:38.134118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f2702d84-02c8-4f04-8917-69b5a02a9f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:38.172984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:38.198636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2702d84-02c8-4f04-8917-69b5a02a9f35 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:38.202680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4516f0-f5ae-498e-8cc2-61327b894f8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:38.203435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4516f0-f5ae-498e-8cc2-61327b894f8d service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:38.203853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4516f0-f5ae-498e-8cc2-61327b894f8d service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:38.204249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4516f0-f5ae-498e-8cc2-61327b894f8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:38.215122 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4516f0-f5ae-498e-8cc2-61327b894f8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:38.216599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4516f0-f5ae-498e-8cc2-61327b894f8d service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:38.228330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 996/1980] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:38.318367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714761b-450a-4fde-aac7-9882648c0151 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:38.319772 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7714761b-450a-4fde-aac7-9882648c0151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:38.331819 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714761b-450a-4fde-aac7-9882648c0151 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:38.336228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c67bc-578f-49bf-b0e0-19ca86e86edd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:38.336831 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c67bc-578f-49bf-b0e0-19ca86e86edd service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:38.337148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c67bc-578f-49bf-b0e0-19ca86e86edd service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:38.337422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c67bc-578f-49bf-b0e0-19ca86e86edd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:38.396624 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c67bc-578f-49bf-b0e0-19ca86e86edd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:38.398604 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c67bc-578f-49bf-b0e0-19ca86e86edd service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:38.407917 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 985/1981] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:38.676743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4516f0-f5ae-498e-8cc2-61327b894f8d service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:38.748942 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4516f0-f5ae-498e-8cc2-61327b894f8d service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:38.753420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd8795-c73f-40bc-9a67-92b61e5bd4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:38.754298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd8795-c73f-40bc-9a67-92b61e5bd4d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:38.754583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd8795-c73f-40bc-9a67-92b61e5bd4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:38.754858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcd8795-c73f-40bc-9a67-92b61e5bd4d7 admin admin] PATH_INFO: `/v3/users/a8376dde1ab64dbdb1f6c37de2ee32be` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:38.756057 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd8795-c73f-40bc-9a67-92b61e5bd4d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8376dde1ab64dbdb1f6c37de2ee32be)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:38.758556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcd8795-c73f-40bc-9a67-92b61e5bd4d7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:38.804089 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fbcd8795-c73f-40bc-9a67-92b61e5bd4d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8376dde1ab64dbdb1f6c37de2ee32be'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:38.804741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fbcd8795-c73f-40bc-9a67-92b61e5bd4d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8376dde1ab64dbdb1f6c37de2ee32be'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:38.816278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fbcd8795-c73f-40bc-9a67-92b61e5bd4d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8376dde1ab64dbdb1f6c37de2ee32be'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:38.823696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fbcd8795-c73f-40bc-9a67-92b61e5bd4d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8376dde1ab64dbdb1f6c37de2ee32be'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:38.830499 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fbcd8795-c73f-40bc-9a67-92b61e5bd4d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8376dde1ab64dbdb1f6c37de2ee32be'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:38.837367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 997/1982] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:38 2022] DELETE /identity/v3/users/a8376dde1ab64dbdb1f6c37de2ee32be => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:38.842275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed316652-6756-4ea2-8cbb-b449302eed98 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:38.842729 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed316652-6756-4ea2-8cbb-b449302eed98 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:38.843088 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed316652-6756-4ea2-8cbb-b449302eed98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:38.860841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ed316652-6756-4ea2-8cbb-b449302eed98 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:38.862072 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ed316652-6756-4ea2-8cbb-b449302eed98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:38.922255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 986/1983] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:38.932613 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd8795-c73f-40bc-9a67-92b61e5bd4d7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:38.934061 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fbcd8795-c73f-40bc-9a67-92b61e5bd4d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:38.992341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcd8795-c73f-40bc-9a67-92b61e5bd4d7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:38.995944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356418a-5d15-4772-b16c-20b10c59dad9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:38.996591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356418a-5d15-4772-b16c-20b10c59dad9 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:38.997073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356418a-5d15-4772-b16c-20b10c59dad9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:38.997545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356418a-5d15-4772-b16c-20b10c59dad9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:39.007735 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356418a-5d15-4772-b16c-20b10c59dad9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:39.009193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356418a-5d15-4772-b16c-20b10c59dad9 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:39.013032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:39.020508 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 998/1984] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:14:39.302192 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed316652-6756-4ea2-8cbb-b449302eed98 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:39.304323 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ed316652-6756-4ea2-8cbb-b449302eed98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:39.316895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed316652-6756-4ea2-8cbb-b449302eed98 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:39.322081 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:39.323071 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:39.323444 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:39.323751 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9 admin admin] PATH_INFO: `/v3/projects/31472a4e05bb45b487df47f4f8255789` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:39.325045 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31472a4e05bb45b487df47f4f8255789)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:39.327027 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:39.380140 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31472a4e05bb45b487df47f4f8255789', 'request_id': 'req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:39.386583 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31472a4e05bb45b487df47f4f8255789', 'request_id': 'req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:39.395153 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31472a4e05bb45b487df47f4f8255789', 'request_id': 'req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:39.399503 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:39.411711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9 admin admin] The token cache is being invalidate because project 31472a4e05bb45b487df47f4f8255789 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:39.412000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:39.413205 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:39.418075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 987/1985] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:39 2022] DELETE /identity/v3/projects/31472a4e05bb45b487df47f4f8255789 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:39.419049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356418a-5d15-4772-b16c-20b10c59dad9 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:39.429509 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356418a-5d15-4772-b16c-20b10c59dad9 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:39.487425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91217437-bb98-4a76-b226-24c54b298750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:39.488361 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91217437-bb98-4a76-b226-24c54b298750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:39.488646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91217437-bb98-4a76-b226-24c54b298750 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:39.488973 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91217437-bb98-4a76-b226-24c54b298750 admin admin] PATH_INFO: `/v3/users/4915cba7c6c644d5ae26615a66f326ff` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:39.490172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91217437-bb98-4a76-b226-24c54b298750 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4915cba7c6c644d5ae26615a66f326ff)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:39.492745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91217437-bb98-4a76-b226-24c54b298750 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:39.541290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91217437-bb98-4a76-b226-24c54b298750 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4915cba7c6c644d5ae26615a66f326ff'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:39.541938 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91217437-bb98-4a76-b226-24c54b298750 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4915cba7c6c644d5ae26615a66f326ff'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:39.554538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91217437-bb98-4a76-b226-24c54b298750 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4915cba7c6c644d5ae26615a66f326ff'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:39.561208 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91217437-bb98-4a76-b226-24c54b298750 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4915cba7c6c644d5ae26615a66f326ff'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:39.568566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91217437-bb98-4a76-b226-24c54b298750 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4915cba7c6c644d5ae26615a66f326ff'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:39.574472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 999/1986] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:39 2022] DELETE /identity/v3/users/4915cba7c6c644d5ae26615a66f326ff => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:39.845451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:39.846611 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:39.867853 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee530e3d-3725-424e-b1cb-8ae6c4ff92d9 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:39.933908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f84327-0743-4f10-b218-0c73cc63c56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:39.934849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f84327-0743-4f10-b218-0c73cc63c56b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:39.935033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f84327-0743-4f10-b218-0c73cc63c56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:39.935438 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f84327-0743-4f10-b218-0c73cc63c56b admin admin] PATH_INFO: `/v3/projects/e1d339ff2a7742a0be32c4c4e2c12090` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:39.937365 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f84327-0743-4f10-b218-0c73cc63c56b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1d339ff2a7742a0be32c4c4e2c12090)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:39.939529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f84327-0743-4f10-b218-0c73cc63c56b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:39.991308 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-35f84327-0743-4f10-b218-0c73cc63c56b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1d339ff2a7742a0be32c4c4e2c12090', 'request_id': 'req-35f84327-0743-4f10-b218-0c73cc63c56b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:39.997263 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-35f84327-0743-4f10-b218-0c73cc63c56b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1d339ff2a7742a0be32c4c4e2c12090', 'request_id': 'req-35f84327-0743-4f10-b218-0c73cc63c56b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:40.004509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-35f84327-0743-4f10-b218-0c73cc63c56b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1d339ff2a7742a0be32c4c4e2c12090', 'request_id': 'req-35f84327-0743-4f10-b218-0c73cc63c56b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:40.014695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-35f84327-0743-4f10-b218-0c73cc63c56b admin admin] The token cache is being invalidate because project e1d339ff2a7742a0be32c4c4e2c12090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:40.014986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-35f84327-0743-4f10-b218-0c73cc63c56b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:40.015611 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-35f84327-0743-4f10-b218-0c73cc63c56b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:40.017288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 988/1987] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:39 2022] DELETE /identity/v3/projects/e1d339ff2a7742a0be32c4c4e2c12090 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:40.024435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84676d0f-07aa-4863-91a3-e1bccc2263d0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:40.024899 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84676d0f-07aa-4863-91a3-e1bccc2263d0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:40.025049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84676d0f-07aa-4863-91a3-e1bccc2263d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:40.043916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-84676d0f-07aa-4863-91a3-e1bccc2263d0 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:40.045156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-84676d0f-07aa-4863-91a3-e1bccc2263d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:40.113270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1000/1988] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:40.121200 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb62579-de8b-41db-83b4-c9cca896de8d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:40.122094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb62579-de8b-41db-83b4-c9cca896de8d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:40.122094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb62579-de8b-41db-83b4-c9cca896de8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:40.142415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9fb62579-de8b-41db-83b4-c9cca896de8d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:40.143612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9fb62579-de8b-41db-83b4-c9cca896de8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:40.160678 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 989/1989] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:40.166488 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84676d0f-07aa-4863-91a3-e1bccc2263d0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:40.177796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84676d0f-07aa-4863-91a3-e1bccc2263d0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:40.183576 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50173455-30dd-4f35-8922-c53790ba4ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:40.184959 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50173455-30dd-4f35-8922-c53790ba4ba5 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:40.185150 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50173455-30dd-4f35-8922-c53790ba4ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:40.185522 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50173455-30dd-4f35-8922-c53790ba4ba5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:40.186417 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50173455-30dd-4f35-8922-c53790ba4ba5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:14:40.186825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50173455-30dd-4f35-8922-c53790ba4ba5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:40.188660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50173455-30dd-4f35-8922-c53790ba4ba5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:40.204331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1001/1990] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:14:40.208045 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb62579-de8b-41db-83b4-c9cca896de8d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:40.218266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb62579-de8b-41db-83b4-c9cca896de8d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:40.223501 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372dc66-cb4e-40f4-b415-e63af0e2af11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:40.224168 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372dc66-cb4e-40f4-b415-e63af0e2af11 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:40.224446 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372dc66-cb4e-40f4-b415-e63af0e2af11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:40.224840 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372dc66-cb4e-40f4-b415-e63af0e2af11 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:40.226004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372dc66-cb4e-40f4-b415-e63af0e2af11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:40.227912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d372dc66-cb4e-40f4-b415-e63af0e2af11 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:40.290645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 990/1991] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:40 2022] POST /identity/v3/projects => generated 373 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:14:40.295449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50173455-30dd-4f35-8922-c53790ba4ba5 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:40.303069 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:40.308701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50173455-30dd-4f35-8922-c53790ba4ba5 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:40.405746 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4297645-bd13-4452-9f5e-147222e739e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:40.406458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4297645-bd13-4452-9f5e-147222e739e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:40.406746 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4297645-bd13-4452-9f5e-147222e739e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:40.407023 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4297645-bd13-4452-9f5e-147222e739e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:40.407721 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4297645-bd13-4452-9f5e-147222e739e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:40.409680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4297645-bd13-4452-9f5e-147222e739e9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:40.432961 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1002/1992] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:40 2022] POST /identity/v3/users => generated 287 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:40.435004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372dc66-cb4e-40f4-b415-e63af0e2af11 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:40.447610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372dc66-cb4e-40f4-b415-e63af0e2af11 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:40.453351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a8ac0-f9dd-498d-a119-9ef69cc82c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:40.454058 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a8ac0-f9dd-498d-a119-9ef69cc82c34 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:40.454349 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a8ac0-f9dd-498d-a119-9ef69cc82c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:40.454690 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a8ac0-f9dd-498d-a119-9ef69cc82c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:40.455405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a8ac0-f9dd-498d-a119-9ef69cc82c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:40.457143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a8ac0-f9dd-498d-a119-9ef69cc82c34 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:40.468438 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 991/1993] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:40.472620 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4297645-bd13-4452-9f5e-147222e739e9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:40.482561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4297645-bd13-4452-9f5e-147222e739e9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:40.487137 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5ee1ba-2330-4960-a0da-8a7fc0dcbc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:40.487791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ee1ba-2330-4960-a0da-8a7fc0dcbc41 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:40.488070 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ee1ba-2330-4960-a0da-8a7fc0dcbc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:40.488341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5ee1ba-2330-4960-a0da-8a7fc0dcbc41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:40.489108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ee1ba-2330-4960-a0da-8a7fc0dcbc41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:40.491336 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5ee1ba-2330-4960-a0da-8a7fc0dcbc41 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:40.501507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1003/1994] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:40.505533 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a8ac0-f9dd-498d-a119-9ef69cc82c34 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:40.516166 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a8ac0-f9dd-498d-a119-9ef69cc82c34 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:40.521077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb45606-b275-453c-ba99-2964a851da04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:40.522322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb45606-b275-453c-ba99-2964a851da04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:40.522322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb45606-b275-453c-ba99-2964a851da04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:40.522577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb45606-b275-453c-ba99-2964a851da04 admin admin] PATH_INFO: `/v3/projects/da859b6d410b4c2b995804fa5ffd190f/users/3f37d419e37b4172b27649775a144ca6/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:40.538919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb45606-b275-453c-ba99-2964a851da04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da859b6d410b4c2b995804fa5ffd190f, user_id=3f37d419e37b4172b27649775a144ca6, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:40.541638 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb45606-b275-453c-ba99-2964a851da04 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:40.556127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 992/1995] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:40 2022] PUT /identity/v3/projects/da859b6d410b4c2b995804fa5ffd190f/users/3f37d419e37b4172b27649775a144ca6/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:40.561589 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4f404-1b52-4000-b918-31bdc18f9786 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:40.561930 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4f404-1b52-4000-b918-31bdc18f9786 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:40.562255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f4f404-1b52-4000-b918-31bdc18f9786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:40.581158 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-32f4f404-1b52-4000-b918-31bdc18f9786 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:40.582387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-32f4f404-1b52-4000-b918-31bdc18f9786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:40.647311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1004/1996] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:40.658799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb45606-b275-453c-ba99-2964a851da04 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:40.660588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ceb45606-b275-453c-ba99-2964a851da04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:40.687854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:40.743351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb45606-b275-453c-ba99-2964a851da04 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:40.747118 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e14c5-3f1f-4c93-8a5d-b55927f9d588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:40.747779 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e14c5-3f1f-4c93-8a5d-b55927f9d588 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:40.748057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e14c5-3f1f-4c93-8a5d-b55927f9d588 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:40.748327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1e14c5-3f1f-4c93-8a5d-b55927f9d588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:40.758333 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e14c5-3f1f-4c93-8a5d-b55927f9d588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:40.759831 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1e14c5-3f1f-4c93-8a5d-b55927f9d588 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:40.771210 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 993/1997] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:46.655200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141641e-95e9-4b13-853d-69b37b0d068d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:46.656114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141641e-95e9-4b13-853d-69b37b0d068d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:46.656560 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2141641e-95e9-4b13-853d-69b37b0d068d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:46.700799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2141641e-95e9-4b13-853d-69b37b0d068d None None] MFA Rules not processed for user `3f37d419e37b4172b27649775a144ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:46.702382 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2141641e-95e9-4b13-853d-69b37b0d068d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:46.880422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1005/1998] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:46.881767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e14c5-3f1f-4c93-8a5d-b55927f9d588 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:46.896414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1e14c5-3f1f-4c93-8a5d-b55927f9d588 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:46.909398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6389df-bd04-4889-882a-fbe793e2043f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:46.910362 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6389df-bd04-4889-882a-fbe793e2043f admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:46.910362 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6389df-bd04-4889-882a-fbe793e2043f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:46.910614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6389df-bd04-4889-882a-fbe793e2043f admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:46.911327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6389df-bd04-4889-882a-fbe793e2043f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:46.913813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6389df-bd04-4889-882a-fbe793e2043f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:46.953080 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 994/1999] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:46 2022] POST /identity/v3/projects => generated 373 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:46.956974 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141641e-95e9-4b13-853d-69b37b0d068d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:46.973178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2141641e-95e9-4b13-853d-69b37b0d068d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:46.983381 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:47.036406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c117f-bbfa-4a98-a952-00e8e6b21328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:47.037117 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c117f-bbfa-4a98-a952-00e8e6b21328 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:47.037921 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c117f-bbfa-4a98-a952-00e8e6b21328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:47.038193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c117f-bbfa-4a98-a952-00e8e6b21328 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:47.038892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c117f-bbfa-4a98-a952-00e8e6b21328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:47.040794 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c117f-bbfa-4a98-a952-00e8e6b21328 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:47.064910 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1006/2000] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:46 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:47.069049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6389df-bd04-4889-882a-fbe793e2043f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:47.080256 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6389df-bd04-4889-882a-fbe793e2043f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:47.084871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3954124-186b-45af-912a-32c3e06f63c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:47.085579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3954124-186b-45af-912a-32c3e06f63c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:47.085860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3954124-186b-45af-912a-32c3e06f63c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:47.086134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3954124-186b-45af-912a-32c3e06f63c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:47.086862 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3954124-186b-45af-912a-32c3e06f63c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:47.088460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3954124-186b-45af-912a-32c3e06f63c4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:47.099759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 995/2001] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:47.104820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c117f-bbfa-4a98-a952-00e8e6b21328 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:47.115130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c117f-bbfa-4a98-a952-00e8e6b21328 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:47.120397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1e3e1-242f-48fb-9721-068fd18dcd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:47.121340 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1e3e1-242f-48fb-9721-068fd18dcd2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:47.121624 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1e3e1-242f-48fb-9721-068fd18dcd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:47.121897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1e3e1-242f-48fb-9721-068fd18dcd2d admin admin] PATH_INFO: `/v3/projects/21548439ec194a86bd7691f16ddd716a/users/4faa600a6acd4fb7bc201806e5528037/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:47.137541 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1e3e1-242f-48fb-9721-068fd18dcd2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=21548439ec194a86bd7691f16ddd716a, user_id=4faa600a6acd4fb7bc201806e5528037, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:47.140219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1e3e1-242f-48fb-9721-068fd18dcd2d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:47.160660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1007/2002] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:47 2022] PUT /identity/v3/projects/21548439ec194a86bd7691f16ddd716a/users/4faa600a6acd4fb7bc201806e5528037/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:47.164916 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3954124-186b-45af-912a-32c3e06f63c4 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:47.175683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3954124-186b-45af-912a-32c3e06f63c4 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:47.246418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ed181-4d9d-4e00-a954-90df9009debd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:47.247016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ed181-4d9d-4e00-a954-90df9009debd admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:47.247291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ed181-4d9d-4e00-a954-90df9009debd admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:47.247561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ed181-4d9d-4e00-a954-90df9009debd admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:47.248295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ed181-4d9d-4e00-a954-90df9009debd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:47.249933 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ed181-4d9d-4e00-a954-90df9009debd admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:47.260766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 996/2003] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:14:47.264296 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1e3e1-242f-48fb-9721-068fd18dcd2d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:47.275411 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1e3e1-242f-48fb-9721-068fd18dcd2d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:47.280404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6155d1f4-815e-4868-8cbb-a5c474761ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:47.281412 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155d1f4-815e-4868-8cbb-a5c474761ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:47.281757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155d1f4-815e-4868-8cbb-a5c474761ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:47.282101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155d1f4-815e-4868-8cbb-a5c474761ca1 admin admin] PATH_INFO: `/v3/projects/21548439ec194a86bd7691f16ddd716a/users/4faa600a6acd4fb7bc201806e5528037/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:47.284147 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155d1f4-815e-4868-8cbb-a5c474761ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21548439ec194a86bd7691f16ddd716a, user_id=4faa600a6acd4fb7bc201806e5528037, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:47.286959 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6155d1f4-815e-4868-8cbb-a5c474761ca1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:47.298602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1008/2004] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:47 2022] PUT /identity/v3/projects/21548439ec194a86bd7691f16ddd716a/users/4faa600a6acd4fb7bc201806e5528037/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:51.022378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ed181-4d9d-4e00-a954-90df9009debd admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:51.024630 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ee5ed181-4d9d-4e00-a954-90df9009debd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:51.051935 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ed181-4d9d-4e00-a954-90df9009debd admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:51.203760 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c9092-5fca-455f-9626-103fc12af5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:51.205114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c9092-5fca-455f-9626-103fc12af5f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:51.205557 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c9092-5fca-455f-9626-103fc12af5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:51.205997 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01c9092-5fca-455f-9626-103fc12af5f4 admin admin] PATH_INFO: `/v3/users/1e2e475aa45440faaa336870675cf9e9` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:51.209569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c9092-5fca-455f-9626-103fc12af5f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e2e475aa45440faaa336870675cf9e9)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:51.213373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01c9092-5fca-455f-9626-103fc12af5f4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:51.297026 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:51.350431 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f01c9092-5fca-455f-9626-103fc12af5f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e2e475aa45440faaa336870675cf9e9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:51.353362 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f01c9092-5fca-455f-9626-103fc12af5f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e2e475aa45440faaa336870675cf9e9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:51.397941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f01c9092-5fca-455f-9626-103fc12af5f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e2e475aa45440faaa336870675cf9e9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:51.414563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f01c9092-5fca-455f-9626-103fc12af5f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e2e475aa45440faaa336870675cf9e9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:51.470772 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f01c9092-5fca-455f-9626-103fc12af5f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e2e475aa45440faaa336870675cf9e9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:51.488984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 997/2005] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:14:51 2022] DELETE /identity/v3/users/1e2e475aa45440faaa336870675cf9e9 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:51.491854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fd922-6044-4dff-8d02-d8ab5e789b62 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:51.492900 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fd922-6044-4dff-8d02-d8ab5e789b62 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:51.493988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fd922-6044-4dff-8d02-d8ab5e789b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:51.528653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-581fd922-6044-4dff-8d02-d8ab5e789b62 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:51.530205 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-581fd922-6044-4dff-8d02-d8ab5e789b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:51.750842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1009/2006] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:51.781019 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c9092-5fca-455f-9626-103fc12af5f4 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:51.783214 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f01c9092-5fca-455f-9626-103fc12af5f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:51.905707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01c9092-5fca-455f-9626-103fc12af5f4 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:51.913622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ddacd-c78d-4e4d-ad12-7d1aadeb07a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:51.917607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ddacd-c78d-4e4d-ad12-7d1aadeb07a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:51.918345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ddacd-c78d-4e4d-ad12-7d1aadeb07a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:51.918345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ddacd-c78d-4e4d-ad12-7d1aadeb07a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:51.932480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ddacd-c78d-4e4d-ad12-7d1aadeb07a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:51.936032 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ddacd-c78d-4e4d-ad12-7d1aadeb07a5 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:51.953810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 998/2007] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:52.434389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fd922-6044-4dff-8d02-d8ab5e789b62 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:52.438216 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-581fd922-6044-4dff-8d02-d8ab5e789b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:52.465643 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fd922-6044-4dff-8d02-d8ab5e789b62 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:52.472407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658495d-93cd-4151-b7ea-7ee86faa09cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:52.473321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658495d-93cd-4151-b7ea-7ee86faa09cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:52.473605 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658495d-93cd-4151-b7ea-7ee86faa09cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:52.473930 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658495d-93cd-4151-b7ea-7ee86faa09cd admin admin] PATH_INFO: `/v3/projects/c1ca205e8be148c8b724b1603111ce65` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:52.480045 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658495d-93cd-4151-b7ea-7ee86faa09cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1ca205e8be148c8b724b1603111ce65)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:52.482515 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658495d-93cd-4151-b7ea-7ee86faa09cd admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:52.630525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3658495d-93cd-4151-b7ea-7ee86faa09cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1ca205e8be148c8b724b1603111ce65', 'request_id': 'req-3658495d-93cd-4151-b7ea-7ee86faa09cd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:52.647567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3658495d-93cd-4151-b7ea-7ee86faa09cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1ca205e8be148c8b724b1603111ce65', 'request_id': 'req-3658495d-93cd-4151-b7ea-7ee86faa09cd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:52.661455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:52.663973 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3658495d-93cd-4151-b7ea-7ee86faa09cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1ca205e8be148c8b724b1603111ce65', 'request_id': 'req-3658495d-93cd-4151-b7ea-7ee86faa09cd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:52.687518 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3658495d-93cd-4151-b7ea-7ee86faa09cd admin admin] The token cache is being invalidate because project c1ca205e8be148c8b724b1603111ce65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:14:52.687913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3658495d-93cd-4151-b7ea-7ee86faa09cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:52.688673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3658495d-93cd-4151-b7ea-7ee86faa09cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:14:52.690442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1010/2008] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:14:52 2022] DELETE /identity/v3/projects/c1ca205e8be148c8b724b1603111ce65 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:52.696166 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8bd40-9dff-4789-9911-c81cde994013 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:52.696653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8bd40-9dff-4789-9911-c81cde994013 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:52.696750 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8bd40-9dff-4789-9911-c81cde994013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:52.739068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-97f8bd40-9dff-4789-9911-c81cde994013 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:52.741744 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-97f8bd40-9dff-4789-9911-c81cde994013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:52.844245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 999/2009] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:52.853593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30237f0-729f-4304-9a55-610a3aa2bc2c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:52.853880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30237f0-729f-4304-9a55-610a3aa2bc2c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:52.854212 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30237f0-729f-4304-9a55-610a3aa2bc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:52.874020 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c30237f0-729f-4304-9a55-610a3aa2bc2c None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:52.875680 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c30237f0-729f-4304-9a55-610a3aa2bc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:52.884151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1011/2010] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:52.888646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8bd40-9dff-4789-9911-c81cde994013 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:52.901124 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8bd40-9dff-4789-9911-c81cde994013 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:52.907018 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779a76d-171b-4468-97e5-10d034a34d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:52.907852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779a76d-171b-4468-97e5-10d034a34d4b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:52.908433 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779a76d-171b-4468-97e5-10d034a34d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:52.909020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779a76d-171b-4468-97e5-10d034a34d4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:52.909918 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779a76d-171b-4468-97e5-10d034a34d4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:14:52.910369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779a76d-171b-4468-97e5-10d034a34d4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:52.913109 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779a76d-171b-4468-97e5-10d034a34d4b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:52.927141 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1000/2011] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:14:52.932210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30237f0-729f-4304-9a55-610a3aa2bc2c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:52.942435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30237f0-729f-4304-9a55-610a3aa2bc2c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:52.949051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52fa26-cb50-42eb-a9d4-3691181adc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:52.950283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52fa26-cb50-42eb-a9d4-3691181adc1e admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:52.950574 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52fa26-cb50-42eb-a9d4-3691181adc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:52.950850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f52fa26-cb50-42eb-a9d4-3691181adc1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:52.951562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52fa26-cb50-42eb-a9d4-3691181adc1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:52.953501 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f52fa26-cb50-42eb-a9d4-3691181adc1e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:53.001039 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1012/2012] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:14:52 2022] POST /identity/v3/projects => generated 373 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:14:53.006401 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779a76d-171b-4468-97e5-10d034a34d4b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:53.020985 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779a76d-171b-4468-97e5-10d034a34d4b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:53.069601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:53.103938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7a664-0074-4dd5-a5e8-9f70bada892d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:53.104622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7a664-0074-4dd5-a5e8-9f70bada892d admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:53.104910 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7a664-0074-4dd5-a5e8-9f70bada892d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:53.105217 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7a664-0074-4dd5-a5e8-9f70bada892d admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:53.105921 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7a664-0074-4dd5-a5e8-9f70bada892d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:53.108961 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7a664-0074-4dd5-a5e8-9f70bada892d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:53.139276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1001/2013] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:14:53 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:14:53.143743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52fa26-cb50-42eb-a9d4-3691181adc1e admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:53.160182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f52fa26-cb50-42eb-a9d4-3691181adc1e admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:53.167853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caf013-942f-4d95-8f25-6f11a884d88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:53.168688 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caf013-942f-4d95-8f25-6f11a884d88b admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:53.169016 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caf013-942f-4d95-8f25-6f11a884d88b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:53.169298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87caf013-942f-4d95-8f25-6f11a884d88b admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:53.170047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caf013-942f-4d95-8f25-6f11a884d88b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:53.172775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87caf013-942f-4d95-8f25-6f11a884d88b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:53.188791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1013/2014] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:53.193149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7a664-0074-4dd5-a5e8-9f70bada892d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:53.208160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7a664-0074-4dd5-a5e8-9f70bada892d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:53.220919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e68918-9a94-4c8c-9e06-0f8486449cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:53.222549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e68918-9a94-4c8c-9e06-0f8486449cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:53.222911 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e68918-9a94-4c8c-9e06-0f8486449cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:53.223248 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e68918-9a94-4c8c-9e06-0f8486449cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:53.224205 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e68918-9a94-4c8c-9e06-0f8486449cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:53.228807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e68918-9a94-4c8c-9e06-0f8486449cc3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:53.251444 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1002/2015] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:14:53.255740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caf013-942f-4d95-8f25-6f11a884d88b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:53.267875 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87caf013-942f-4d95-8f25-6f11a884d88b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:53.289779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14095602-420a-4b59-9a75-068a2eb561f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:53.290833 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14095602-420a-4b59-9a75-068a2eb561f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:53.291111 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14095602-420a-4b59-9a75-068a2eb561f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:53.291380 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14095602-420a-4b59-9a75-068a2eb561f6 admin admin] PATH_INFO: `/v3/projects/8f2bfe9f5d124fea8d2c2d328e378199/users/275f5d63bfbc4051bafbfd1c3fb0f9d1/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:53.317375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14095602-420a-4b59-9a75-068a2eb561f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f2bfe9f5d124fea8d2c2d328e378199, user_id=275f5d63bfbc4051bafbfd1c3fb0f9d1, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:53.320619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14095602-420a-4b59-9a75-068a2eb561f6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:53.349689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1014/2016] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:14:53 2022] PUT /identity/v3/projects/8f2bfe9f5d124fea8d2c2d328e378199/users/275f5d63bfbc4051bafbfd1c3fb0f9d1/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:14:53.360909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88d8fe-44ec-408e-ba38-beb8e90b16ea None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:53.361224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88d8fe-44ec-408e-ba38-beb8e90b16ea None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:53.361500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee88d8fe-44ec-408e-ba38-beb8e90b16ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:53.427839 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ee88d8fe-44ec-408e-ba38-beb8e90b16ea None None] MFA Rules not processed for user `275f5d63bfbc4051bafbfd1c3fb0f9d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:53.429184 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ee88d8fe-44ec-408e-ba38-beb8e90b16ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:53.526676 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1003/2017] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:53.545158 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14095602-420a-4b59-9a75-068a2eb561f6 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:53.547474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-14095602-420a-4b59-9a75-068a2eb561f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:53.645152 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14095602-420a-4b59-9a75-068a2eb561f6 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:53.650456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a701ff3-6f9f-4aac-a70e-ea7a8d0ced60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:53.651172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a701ff3-6f9f-4aac-a70e-ea7a8d0ced60 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:53.651602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a701ff3-6f9f-4aac-a70e-ea7a8d0ced60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:53.651969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a701ff3-6f9f-4aac-a70e-ea7a8d0ced60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:53.662205 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a701ff3-6f9f-4aac-a70e-ea7a8d0ced60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:53.663879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a701ff3-6f9f-4aac-a70e-ea7a8d0ced60 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:53.674687 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1015/2018] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:54.723085 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55ed57-5069-447c-8ed1-1d9705ded6d1 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:54.723655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55ed57-5069-447c-8ed1-1d9705ded6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:54.723655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef55ed57-5069-447c-8ed1-1d9705ded6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:54.752211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ef55ed57-5069-447c-8ed1-1d9705ded6d1 None None] MFA Rules not processed for user `4faa600a6acd4fb7bc201806e5528037`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:54.753477 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ef55ed57-5069-447c-8ed1-1d9705ded6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:54.829084 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1004/2019] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:54 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:14:54.842986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a701ff3-6f9f-4aac-a70e-ea7a8d0ced60 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:14:54.844435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7a701ff3-6f9f-4aac-a70e-ea7a8d0ced60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:54.920364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a701ff3-6f9f-4aac-a70e-ea7a8d0ced60 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:14:54.925924 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b63f1b-44b0-4f5d-a0e4-597d08a9c1b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:14:54.926582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b63f1b-44b0-4f5d-a0e4-597d08a9c1b7 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:54.926862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b63f1b-44b0-4f5d-a0e4-597d08a9c1b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:54.927134 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b63f1b-44b0-4f5d-a0e4-597d08a9c1b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:54.929119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b6b63f1b-44b0-4f5d-a0e4-597d08a9c1b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:54.955374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:14:54.993388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b63f1b-44b0-4f5d-a0e4-597d08a9c1b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:14:54.994992 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b63f1b-44b0-4f5d-a0e4-597d08a9c1b7 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:14:55.004809 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1016/2020] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:14:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:14:55.981516 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dfe45-ace7-4638-83ff-a5e3e0b6ead4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:14:55.982003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dfe45-ace7-4638-83ff-a5e3e0b6ead4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:14:55.982104 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9dfe45-ace7-4638-83ff-a5e3e0b6ead4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:14:56.000283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-da9dfe45-ace7-4638-83ff-a5e3e0b6ead4 None None] MFA Rules not processed for user `3f37d419e37b4172b27649775a144ca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:14:56.001506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-da9dfe45-ace7-4638-83ff-a5e3e0b6ead4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:14:56.010004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1005/2021] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:14:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:00.282400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b63f1b-44b0-4f5d-a0e4-597d08a9c1b7 service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:00.436740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b63f1b-44b0-4f5d-a0e4-597d08a9c1b7 service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:00.447793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00277cba-b99f-4118-8150-036d65bf4123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:00.448937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00277cba-b99f-4118-8150-036d65bf4123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:00.449311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00277cba-b99f-4118-8150-036d65bf4123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:00.449658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00277cba-b99f-4118-8150-036d65bf4123 admin admin] PATH_INFO: `/v3/users/275f5d63bfbc4051bafbfd1c3fb0f9d1` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:00.451634 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00277cba-b99f-4118-8150-036d65bf4123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=275f5d63bfbc4051bafbfd1c3fb0f9d1)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:00.454960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00277cba-b99f-4118-8150-036d65bf4123 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:00.572699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-00277cba-b99f-4118-8150-036d65bf4123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '275f5d63bfbc4051bafbfd1c3fb0f9d1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:00.573892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-00277cba-b99f-4118-8150-036d65bf4123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '275f5d63bfbc4051bafbfd1c3fb0f9d1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:00.601122 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-00277cba-b99f-4118-8150-036d65bf4123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '275f5d63bfbc4051bafbfd1c3fb0f9d1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:00.630878 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-00277cba-b99f-4118-8150-036d65bf4123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '275f5d63bfbc4051bafbfd1c3fb0f9d1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:00.647385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-00277cba-b99f-4118-8150-036d65bf4123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '275f5d63bfbc4051bafbfd1c3fb0f9d1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:00.656776 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1017/2022] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:00 2022] DELETE /identity/v3/users/275f5d63bfbc4051bafbfd1c3fb0f9d1 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:00.661838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3df29-ab4a-4ade-8244-7d96909cb474 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:00.663909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3df29-ab4a-4ade-8244-7d96909cb474 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:00.664387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a3df29-ab4a-4ade-8244-7d96909cb474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:00.690382 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e9a3df29-ab4a-4ade-8244-7d96909cb474 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:00.691665 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e9a3df29-ab4a-4ade-8244-7d96909cb474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:00.719276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:00.805158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1006/2023] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:00.813835 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00277cba-b99f-4118-8150-036d65bf4123 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:00.815957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-00277cba-b99f-4118-8150-036d65bf4123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:00.870891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:00.883517 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00277cba-b99f-4118-8150-036d65bf4123 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:00.889064 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80b60f6-b1d1-4e0e-af12-a211a950f63d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:00.890968 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b60f6-b1d1-4e0e-af12-a211a950f63d service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:00.891486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b60f6-b1d1-4e0e-af12-a211a950f63d service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:00.891858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80b60f6-b1d1-4e0e-af12-a211a950f63d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:00.903831 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b60f6-b1d1-4e0e-af12-a211a950f63d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:00.907198 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80b60f6-b1d1-4e0e-af12-a211a950f63d service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:00.922034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1018/2024] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:15:01.280344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3df29-ab4a-4ade-8244-7d96909cb474 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:01.282921 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e9a3df29-ab4a-4ade-8244-7d96909cb474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:01.295534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a3df29-ab4a-4ade-8244-7d96909cb474 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:01.301816 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d1708-b8c9-4edf-8de1-4b44e8775189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:01.305539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d1708-b8c9-4edf-8de1-4b44e8775189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:01.305539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d1708-b8c9-4edf-8de1-4b44e8775189 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:01.305539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d1708-b8c9-4edf-8de1-4b44e8775189 admin admin] PATH_INFO: `/v3/projects/8f2bfe9f5d124fea8d2c2d328e378199` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:01.305539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d1708-b8c9-4edf-8de1-4b44e8775189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f2bfe9f5d124fea8d2c2d328e378199)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:01.306928 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d1708-b8c9-4edf-8de1-4b44e8775189 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:01.360330 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1c3d1708-b8c9-4edf-8de1-4b44e8775189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f2bfe9f5d124fea8d2c2d328e378199', 'request_id': 'req-1c3d1708-b8c9-4edf-8de1-4b44e8775189'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:01.367003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1c3d1708-b8c9-4edf-8de1-4b44e8775189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f2bfe9f5d124fea8d2c2d328e378199', 'request_id': 'req-1c3d1708-b8c9-4edf-8de1-4b44e8775189'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:01.375000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1c3d1708-b8c9-4edf-8de1-4b44e8775189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f2bfe9f5d124fea8d2c2d328e378199', 'request_id': 'req-1c3d1708-b8c9-4edf-8de1-4b44e8775189'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:01.386439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1c3d1708-b8c9-4edf-8de1-4b44e8775189 admin admin] The token cache is being invalidate because project 8f2bfe9f5d124fea8d2c2d328e378199 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:01.386735 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1c3d1708-b8c9-4edf-8de1-4b44e8775189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:01.387389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1c3d1708-b8c9-4edf-8de1-4b44e8775189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:01.394977 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1007/2025] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:01 2022] DELETE /identity/v3/projects/8f2bfe9f5d124fea8d2c2d328e378199 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:01.400745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00986b-b7ad-4af4-b566-0eaeab280c03 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:01.401249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00986b-b7ad-4af4-b566-0eaeab280c03 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:01.401347 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f00986b-b7ad-4af4-b566-0eaeab280c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:01.423220 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7f00986b-b7ad-4af4-b566-0eaeab280c03 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:01.424444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7f00986b-b7ad-4af4-b566-0eaeab280c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:01.489976 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1019/2026] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:01.499308 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab034cef-ed74-42a5-920c-7078459a1b74 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:01.499591 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab034cef-ed74-42a5-920c-7078459a1b74 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:01.499866 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab034cef-ed74-42a5-920c-7078459a1b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:01.522585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ab034cef-ed74-42a5-920c-7078459a1b74 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:01.523787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ab034cef-ed74-42a5-920c-7078459a1b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:01.533319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1008/2027] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:01.537650 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00986b-b7ad-4af4-b566-0eaeab280c03 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:01.547570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f00986b-b7ad-4af4-b566-0eaeab280c03 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:01.552179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48667bc4-7e31-4312-a68d-747d8dd62ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:01.553434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48667bc4-7e31-4312-a68d-747d8dd62ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:01.553716 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48667bc4-7e31-4312-a68d-747d8dd62ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:01.553989 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48667bc4-7e31-4312-a68d-747d8dd62ec9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:01.554727 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48667bc4-7e31-4312-a68d-747d8dd62ec9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:15:01.555033 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48667bc4-7e31-4312-a68d-747d8dd62ec9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:01.556636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48667bc4-7e31-4312-a68d-747d8dd62ec9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:01.571046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1020/2028] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:15:01.575371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab034cef-ed74-42a5-920c-7078459a1b74 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:01.585002 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab034cef-ed74-42a5-920c-7078459a1b74 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:01.589743 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705d9c9-6c74-4cfb-8bac-218ebfb4b536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:01.590416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705d9c9-6c74-4cfb-8bac-218ebfb4b536 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:01.590696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705d9c9-6c74-4cfb-8bac-218ebfb4b536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:01.590968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705d9c9-6c74-4cfb-8bac-218ebfb4b536 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:01.591677 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705d9c9-6c74-4cfb-8bac-218ebfb4b536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:01.593638 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7705d9c9-6c74-4cfb-8bac-218ebfb4b536 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:01.626017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1009/2029] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:01 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:01.630197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48667bc4-7e31-4312-a68d-747d8dd62ec9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:01.640869 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48667bc4-7e31-4312-a68d-747d8dd62ec9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:01.702558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072bb1b-38d1-422d-b69f-319b179e206d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:01.703331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072bb1b-38d1-422d-b69f-319b179e206d admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:01.703678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072bb1b-38d1-422d-b69f-319b179e206d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:01.704015 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072bb1b-38d1-422d-b69f-319b179e206d admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:01.704786 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072bb1b-38d1-422d-b69f-319b179e206d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:01.707209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072bb1b-38d1-422d-b69f-319b179e206d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:01.731694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1021/2030] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:01 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:01.733318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705d9c9-6c74-4cfb-8bac-218ebfb4b536 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:01.745369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705d9c9-6c74-4cfb-8bac-218ebfb4b536 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:01.750619 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ce07e-ec24-4712-aa79-9b52c7482653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:01.751303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ce07e-ec24-4712-aa79-9b52c7482653 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:01.751581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ce07e-ec24-4712-aa79-9b52c7482653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:01.752036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ce07e-ec24-4712-aa79-9b52c7482653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:01.752943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ce07e-ec24-4712-aa79-9b52c7482653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:01.755266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ce07e-ec24-4712-aa79-9b52c7482653 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:01.769328 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1010/2031] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:01.773899 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072bb1b-38d1-422d-b69f-319b179e206d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:01.788237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072bb1b-38d1-422d-b69f-319b179e206d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:01.799732 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd551cd-5850-4ebe-9d0a-744290a6cc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:01.800442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd551cd-5850-4ebe-9d0a-744290a6cc1c admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:01.800730 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd551cd-5850-4ebe-9d0a-744290a6cc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:01.801061 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd551cd-5850-4ebe-9d0a-744290a6cc1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:01.801813 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd551cd-5850-4ebe-9d0a-744290a6cc1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:01.805774 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd551cd-5850-4ebe-9d0a-744290a6cc1c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:01.824042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1022/2032] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:01.825013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ce07e-ec24-4712-aa79-9b52c7482653 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:01.838816 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ce07e-ec24-4712-aa79-9b52c7482653 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:01.843744 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509af8d-bbef-468f-be67-ae504973f5ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:01.844639 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509af8d-bbef-468f-be67-ae504973f5ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:01.844934 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509af8d-bbef-468f-be67-ae504973f5ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:01.845299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509af8d-bbef-468f-be67-ae504973f5ee admin admin] PATH_INFO: `/v3/projects/7d8c14d0efbc46489509f02fed9265fe/users/e1f3c3ccaa754cff9f9b5326f8f29cfd/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:01.863912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509af8d-bbef-468f-be67-ae504973f5ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d8c14d0efbc46489509f02fed9265fe, user_id=e1f3c3ccaa754cff9f9b5326f8f29cfd, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:01.867358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509af8d-bbef-468f-be67-ae504973f5ee admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:01.885157 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1011/2033] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:01 2022] PUT /identity/v3/projects/7d8c14d0efbc46489509f02fed9265fe/users/e1f3c3ccaa754cff9f9b5326f8f29cfd/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:01.895280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1294dcc-c10b-4f9a-b4b0-3747c30394e3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:01.895574 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1294dcc-c10b-4f9a-b4b0-3747c30394e3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:01.895855 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1294dcc-c10b-4f9a-b4b0-3747c30394e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:01.923721 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f1294dcc-c10b-4f9a-b4b0-3747c30394e3 None None] MFA Rules not processed for user `e1f3c3ccaa754cff9f9b5326f8f29cfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:01.925148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f1294dcc-c10b-4f9a-b4b0-3747c30394e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:01.970573 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:02.011014 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1023/2034] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:02.022595 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509af8d-bbef-468f-be67-ae504973f5ee admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:02.024394 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a509af8d-bbef-468f-be67-ae504973f5ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:02.087440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:02.102192 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509af8d-bbef-468f-be67-ae504973f5ee admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:02.107419 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7176f5-e843-46bc-8df1-645fee6c7448 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:02.108084 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7176f5-e843-46bc-8df1-645fee6c7448 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:02.108366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7176f5-e843-46bc-8df1-645fee6c7448 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:02.111146 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7176f5-e843-46bc-8df1-645fee6c7448 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:02.121203 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7176f5-e843-46bc-8df1-645fee6c7448 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:02.122652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7176f5-e843-46bc-8df1-645fee6c7448 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:02.135012 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1012/2035] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:10.048531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1294dcc-c10b-4f9a-b4b0-3747c30394e3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:10.050155 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f1294dcc-c10b-4f9a-b4b0-3747c30394e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:10.135159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1294dcc-c10b-4f9a-b4b0-3747c30394e3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:10.141546 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414e7df-aba1-40b9-992d-237b52226d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:10.142534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414e7df-aba1-40b9-992d-237b52226d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:10.142818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414e7df-aba1-40b9-992d-237b52226d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:10.143156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414e7df-aba1-40b9-992d-237b52226d6a admin admin] PATH_INFO: `/v3/users/68923614145943ba85d77d9d4a84ca72` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:10.144311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414e7df-aba1-40b9-992d-237b52226d6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=68923614145943ba85d77d9d4a84ca72)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:10.146268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414e7df-aba1-40b9-992d-237b52226d6a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:10.195619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1414e7df-aba1-40b9-992d-237b52226d6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68923614145943ba85d77d9d4a84ca72'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:10.196430 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1414e7df-aba1-40b9-992d-237b52226d6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68923614145943ba85d77d9d4a84ca72'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:10.212322 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1414e7df-aba1-40b9-992d-237b52226d6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68923614145943ba85d77d9d4a84ca72'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:10.219184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1414e7df-aba1-40b9-992d-237b52226d6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68923614145943ba85d77d9d4a84ca72'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:10.226778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1414e7df-aba1-40b9-992d-237b52226d6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68923614145943ba85d77d9d4a84ca72'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:10.237025 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1024/2036] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:10 2022] DELETE /identity/v3/users/68923614145943ba85d77d9d4a84ca72 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:10.244499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e0dd8-f649-4b84-893b-d0a1ca0ef141 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:10.244996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e0dd8-f649-4b84-893b-d0a1ca0ef141 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:10.246592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e0dd8-f649-4b84-893b-d0a1ca0ef141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:10.267412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-736e0dd8-f649-4b84-893b-d0a1ca0ef141 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:10.268622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-736e0dd8-f649-4b84-893b-d0a1ca0ef141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:10.331566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1013/2037] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:10.343527 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414e7df-aba1-40b9-992d-237b52226d6a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:10.344969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1414e7df-aba1-40b9-992d-237b52226d6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:10.416306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414e7df-aba1-40b9-992d-237b52226d6a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:10.420246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4fa1dc-652f-4855-9d21-d73db03ca8d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:10.421130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fa1dc-652f-4855-9d21-d73db03ca8d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:10.421497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fa1dc-652f-4855-9d21-d73db03ca8d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:10.421776 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4fa1dc-652f-4855-9d21-d73db03ca8d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:10.431633 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4fa1dc-652f-4855-9d21-d73db03ca8d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:10.433159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4fa1dc-652f-4855-9d21-d73db03ca8d5 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:10.443542 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1025/2038] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:10.726930 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e0dd8-f649-4b84-893b-d0a1ca0ef141 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:10.728489 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-736e0dd8-f649-4b84-893b-d0a1ca0ef141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:10.741248 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e0dd8-f649-4b84-893b-d0a1ca0ef141 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:10.746081 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af9a77-e016-43bf-96b0-473239348d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:10.746949 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af9a77-e016-43bf-96b0-473239348d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:10.747230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af9a77-e016-43bf-96b0-473239348d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:10.747501 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af9a77-e016-43bf-96b0-473239348d3d admin admin] PATH_INFO: `/v3/projects/49d652469bad4f92ad11f2fb3b50a00c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:10.748671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af9a77-e016-43bf-96b0-473239348d3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=49d652469bad4f92ad11f2fb3b50a00c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:10.750757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af9a77-e016-43bf-96b0-473239348d3d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:10.797607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-22af9a77-e016-43bf-96b0-473239348d3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49d652469bad4f92ad11f2fb3b50a00c', 'request_id': 'req-22af9a77-e016-43bf-96b0-473239348d3d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:10.803578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-22af9a77-e016-43bf-96b0-473239348d3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49d652469bad4f92ad11f2fb3b50a00c', 'request_id': 'req-22af9a77-e016-43bf-96b0-473239348d3d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:10.810667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-22af9a77-e016-43bf-96b0-473239348d3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49d652469bad4f92ad11f2fb3b50a00c', 'request_id': 'req-22af9a77-e016-43bf-96b0-473239348d3d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:10.822196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-22af9a77-e016-43bf-96b0-473239348d3d admin admin] The token cache is being invalidate because project 49d652469bad4f92ad11f2fb3b50a00c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:10.822480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-22af9a77-e016-43bf-96b0-473239348d3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:10.823129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-22af9a77-e016-43bf-96b0-473239348d3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:10.824696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1014/2039] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:10 2022] DELETE /identity/v3/projects/49d652469bad4f92ad11f2fb3b50a00c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:10.829763 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805ab55-8090-4777-a2a3-aef3a35e7b97 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:10.830081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805ab55-8090-4777-a2a3-aef3a35e7b97 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:10.830359 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805ab55-8090-4777-a2a3-aef3a35e7b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:10.838256 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:10.850169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c805ab55-8090-4777-a2a3-aef3a35e7b97 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:10.851839 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c805ab55-8090-4777-a2a3-aef3a35e7b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:10.911356 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1026/2040] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:10.919125 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c3543-d1a4-435e-bb33-3c2123477bc0 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:10.919410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c3543-d1a4-435e-bb33-3c2123477bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:10.919688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c3543-d1a4-435e-bb33-3c2123477bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:10.937911 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2e4c3543-d1a4-435e-bb33-3c2123477bc0 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:10.939110 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2e4c3543-d1a4-435e-bb33-3c2123477bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:10.947767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1015/2041] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:10.952065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805ab55-8090-4777-a2a3-aef3a35e7b97 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:10.963789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805ab55-8090-4777-a2a3-aef3a35e7b97 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:10.968512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b496b-dad8-4b6b-a697-be60f6941432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:10.969287 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b496b-dad8-4b6b-a697-be60f6941432 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:10.969479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b496b-dad8-4b6b-a697-be60f6941432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:10.969757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b496b-dad8-4b6b-a697-be60f6941432 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:10.970502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b496b-dad8-4b6b-a697-be60f6941432 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:15:10.970810 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b496b-dad8-4b6b-a697-be60f6941432 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:10.972407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b496b-dad8-4b6b-a697-be60f6941432 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:10.987940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1027/2042] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:15:10.989004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c3543-d1a4-435e-bb33-3c2123477bc0 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:10.998630 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c3543-d1a4-435e-bb33-3c2123477bc0 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:11.003632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e301a-bf30-4167-89d4-50c6a3b35345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:11.004299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e301a-bf30-4167-89d4-50c6a3b35345 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:11.004584 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e301a-bf30-4167-89d4-50c6a3b35345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:11.004864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9e301a-bf30-4167-89d4-50c6a3b35345 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:11.005644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e301a-bf30-4167-89d4-50c6a3b35345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:11.008078 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9e301a-bf30-4167-89d4-50c6a3b35345 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:11.036155 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1016/2043] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:10 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:15:11.040761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b496b-dad8-4b6b-a697-be60f6941432 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:11.051743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b496b-dad8-4b6b-a697-be60f6941432 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:11.122081 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e301a-bf30-4167-89d4-50c6a3b35345 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:11.124500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-bc9e301a-bf30-4167-89d4-50c6a3b35345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:11.139563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2b9fa-a8d5-4bf3-bc37-cfc8b1aae311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:11.140249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2b9fa-a8d5-4bf3-bc37-cfc8b1aae311 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:11.140535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2b9fa-a8d5-4bf3-bc37-cfc8b1aae311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:11.140814 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2b9fa-a8d5-4bf3-bc37-cfc8b1aae311 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:11.141561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2b9fa-a8d5-4bf3-bc37-cfc8b1aae311 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:11.143688 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2b9fa-a8d5-4bf3-bc37-cfc8b1aae311 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:11.181685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1028/2044] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:11 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:11.189541 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:11.193440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2b9fa-a8d5-4bf3-bc37-cfc8b1aae311 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:11.208972 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2b9fa-a8d5-4bf3-bc37-cfc8b1aae311 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:11.215682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7f444-a88f-41e1-b9f4-1eb4d24d054a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:11.216591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7f444-a88f-41e1-b9f4-1eb4d24d054a admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:11.217059 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7f444-a88f-41e1-b9f4-1eb4d24d054a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:11.217485 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b7f444-a88f-41e1-b9f4-1eb4d24d054a admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:11.218377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7f444-a88f-41e1-b9f4-1eb4d24d054a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:11.220290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b7f444-a88f-41e1-b9f4-1eb4d24d054a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:11.222973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9e301a-bf30-4167-89d4-50c6a3b35345 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:11.229302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575baa88-ffee-470e-a0fc-bc4c1d05b93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:11.230504 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575baa88-ffee-470e-a0fc-bc4c1d05b93d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:11.230945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575baa88-ffee-470e-a0fc-bc4c1d05b93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:11.231315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575baa88-ffee-470e-a0fc-bc4c1d05b93d admin admin] PATH_INFO: `/v3/users/e1f3c3ccaa754cff9f9b5326f8f29cfd` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:11.232885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575baa88-ffee-470e-a0fc-bc4c1d05b93d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1f3c3ccaa754cff9f9b5326f8f29cfd)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:11.235321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575baa88-ffee-470e-a0fc-bc4c1d05b93d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:11.237044 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1029/2045] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:11.241521 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7f444-a88f-41e1-b9f4-1eb4d24d054a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:11.254727 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b7f444-a88f-41e1-b9f4-1eb4d24d054a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:11.262349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257e8c2-656d-4a99-b1c5-84a0bb3eb3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:11.263174 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257e8c2-656d-4a99-b1c5-84a0bb3eb3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:11.263461 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257e8c2-656d-4a99-b1c5-84a0bb3eb3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:11.264290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257e8c2-656d-4a99-b1c5-84a0bb3eb3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:11.265279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257e8c2-656d-4a99-b1c5-84a0bb3eb3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:11.267169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257e8c2-656d-4a99-b1c5-84a0bb3eb3c1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:11.284391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1030/2046] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:11.285579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257e8c2-656d-4a99-b1c5-84a0bb3eb3c1 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:11.301856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257e8c2-656d-4a99-b1c5-84a0bb3eb3c1 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:11.311451 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb87773d-03d3-441b-af6e-068fc7488057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:11.312872 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87773d-03d3-441b-af6e-068fc7488057 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:11.313222 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87773d-03d3-441b-af6e-068fc7488057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:11.313719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb87773d-03d3-441b-af6e-068fc7488057 admin admin] PATH_INFO: `/v3/projects/f523b9be668a4dc69619027cec280a08/users/5298be027cf843638957c4a15003db7f/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:11.323257 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-575baa88-ffee-470e-a0fc-bc4c1d05b93d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1f3c3ccaa754cff9f9b5326f8f29cfd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:11.325201 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-575baa88-ffee-470e-a0fc-bc4c1d05b93d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1f3c3ccaa754cff9f9b5326f8f29cfd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:11.341977 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87773d-03d3-441b-af6e-068fc7488057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f523b9be668a4dc69619027cec280a08, user_id=5298be027cf843638957c4a15003db7f, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:11.345677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb87773d-03d3-441b-af6e-068fc7488057 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:11.346480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-575baa88-ffee-470e-a0fc-bc4c1d05b93d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1f3c3ccaa754cff9f9b5326f8f29cfd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:11.355516 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-575baa88-ffee-470e-a0fc-bc4c1d05b93d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1f3c3ccaa754cff9f9b5326f8f29cfd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:11.366936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-575baa88-ffee-470e-a0fc-bc4c1d05b93d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1f3c3ccaa754cff9f9b5326f8f29cfd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:11.372969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1031/2047] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:11 2022] PUT /identity/v3/projects/f523b9be668a4dc69619027cec280a08/users/5298be027cf843638957c4a15003db7f/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:11.376723 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfc43b-387a-4722-aec7-64f3126e7b63 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:11.381729 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfc43b-387a-4722-aec7-64f3126e7b63 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:11.382247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbfc43b-387a-4722-aec7-64f3126e7b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:11.388425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1017/2048] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:11 2022] DELETE /identity/v3/users/e1f3c3ccaa754cff9f9b5326f8f29cfd => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:11.392545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0d626-8530-43aa-ba1b-b4ccdf20d8f2 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:11.393033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0d626-8530-43aa-ba1b-b4ccdf20d8f2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:11.393472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0d626-8530-43aa-ba1b-b4ccdf20d8f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:11.408535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1cbfc43b-387a-4722-aec7-64f3126e7b63 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:11.409753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1cbfc43b-387a-4722-aec7-64f3126e7b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:11.410838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a6e0d626-8530-43aa-ba1b-b4ccdf20d8f2 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:11.412004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a6e0d626-8530-43aa-ba1b-b4ccdf20d8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:11.458269 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:11.509067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1032/2049] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:11.509410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1018/2050] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:15:11.518955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfc43b-387a-4722-aec7-64f3126e7b63 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:11.520452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1cbfc43b-387a-4722-aec7-64f3126e7b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:11.522913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0d626-8530-43aa-ba1b-b4ccdf20d8f2 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:11.524967 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a6e0d626-8530-43aa-ba1b-b4ccdf20d8f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:11.595749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbfc43b-387a-4722-aec7-64f3126e7b63 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:11.600492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b0445-9621-419f-a83c-2ab81bd12d2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:11.601565 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b0445-9621-419f-a83c-2ab81bd12d2e service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:11.601836 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b0445-9621-419f-a83c-2ab81bd12d2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:11.602106 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b0445-9621-419f-a83c-2ab81bd12d2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:11.619922 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b0445-9621-419f-a83c-2ab81bd12d2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:11.620393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0d626-8530-43aa-ba1b-b4ccdf20d8f2 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:11.622292 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b0445-9621-419f-a83c-2ab81bd12d2e service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:11.624512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380daae-32cd-40e0-8313-248d324c6bc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:11.625314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380daae-32cd-40e0-8313-248d324c6bc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:11.625585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380daae-32cd-40e0-8313-248d324c6bc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:11.626137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3380daae-32cd-40e0-8313-248d324c6bc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:11.639674 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380daae-32cd-40e0-8313-248d324c6bc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:11.641314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3380daae-32cd-40e0-8313-248d324c6bc6 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:11.654458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1019/2051] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:11.657891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1033/2052] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:12.026296 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380daae-32cd-40e0-8313-248d324c6bc6 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:12.028221 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3380daae-32cd-40e0-8313-248d324c6bc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:12.037126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:12.044379 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3380daae-32cd-40e0-8313-248d324c6bc6 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:12.051759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8589c4-0dbd-468f-960d-168df47b8b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:12.053208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8589c4-0dbd-468f-960d-168df47b8b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:12.053806 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8589c4-0dbd-468f-960d-168df47b8b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:12.054345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8589c4-0dbd-468f-960d-168df47b8b4c admin admin] PATH_INFO: `/v3/projects/7d8c14d0efbc46489509f02fed9265fe` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:12.056158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8589c4-0dbd-468f-960d-168df47b8b4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d8c14d0efbc46489509f02fed9265fe)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:12.058947 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8589c4-0dbd-468f-960d-168df47b8b4c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:12.121666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a8589c4-0dbd-468f-960d-168df47b8b4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d8c14d0efbc46489509f02fed9265fe', 'request_id': 'req-1a8589c4-0dbd-468f-960d-168df47b8b4c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:12.129264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a8589c4-0dbd-468f-960d-168df47b8b4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d8c14d0efbc46489509f02fed9265fe', 'request_id': 'req-1a8589c4-0dbd-468f-960d-168df47b8b4c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:12.140138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a8589c4-0dbd-468f-960d-168df47b8b4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d8c14d0efbc46489509f02fed9265fe', 'request_id': 'req-1a8589c4-0dbd-468f-960d-168df47b8b4c'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:12.156431 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a8589c4-0dbd-468f-960d-168df47b8b4c admin admin] The token cache is being invalidate because project 7d8c14d0efbc46489509f02fed9265fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:12.156773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a8589c4-0dbd-468f-960d-168df47b8b4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:12.157471 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a8589c4-0dbd-468f-960d-168df47b8b4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:12.159390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1020/2053] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:12 2022] DELETE /identity/v3/projects/7d8c14d0efbc46489509f02fed9265fe => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:12.430439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da435e-e119-4172-82c5-857dbf2a7185 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:12.430954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da435e-e119-4172-82c5-857dbf2a7185 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:12.430954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da435e-e119-4172-82c5-857dbf2a7185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:12.473807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f3da435e-e119-4172-82c5-857dbf2a7185 None None] MFA Rules not processed for user `5298be027cf843638957c4a15003db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:12.475114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f3da435e-e119-4172-82c5-857dbf2a7185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:12.601434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1034/2054] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:12.611931 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8589c4-0dbd-468f-960d-168df47b8b4c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:12.613518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1a8589c4-0dbd-468f-960d-168df47b8b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:12.691338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8589c4-0dbd-468f-960d-168df47b8b4c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:12.696909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbf1d5-42d5-45bc-adfd-aac6ee4ce7e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:12.697973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbf1d5-42d5-45bc-adfd-aac6ee4ce7e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:12.697973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbf1d5-42d5-45bc-adfd-aac6ee4ce7e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:12.698350 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbbf1d5-42d5-45bc-adfd-aac6ee4ce7e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:12.707731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbf1d5-42d5-45bc-adfd-aac6ee4ce7e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:12.709429 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbbf1d5-42d5-45bc-adfd-aac6ee4ce7e7 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:12.726678 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1021/2055] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:14.149799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badc74f-4e5f-42d1-b8dc-d6c4c00fe79b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:14.150421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badc74f-4e5f-42d1-b8dc-d6c4c00fe79b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:14.150421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9badc74f-4e5f-42d1-b8dc-d6c4c00fe79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:14.185788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-9badc74f-4e5f-42d1-b8dc-d6c4c00fe79b None None] MFA Rules not processed for user `5298be027cf843638957c4a15003db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:14.187109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9badc74f-4e5f-42d1-b8dc-d6c4c00fe79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:14.209115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1035/2056] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:15:16.923327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbf1d5-42d5-45bc-adfd-aac6ee4ce7e7 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:16.924769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9dbbf1d5-42d5-45bc-adfd-aac6ee4ce7e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:16.992142 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbbf1d5-42d5-45bc-adfd-aac6ee4ce7e7 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:16.996698 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9886298-5a2d-423e-acb6-da7970089c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:16.997602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9886298-5a2d-423e-acb6-da7970089c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:16.997885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9886298-5a2d-423e-acb6-da7970089c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:16.998159 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9886298-5a2d-423e-acb6-da7970089c63 admin admin] PATH_INFO: `/v3/users/fed44db7dc014719b5ed0399c715435b` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:16.999381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9886298-5a2d-423e-acb6-da7970089c63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fed44db7dc014719b5ed0399c715435b)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:17.001342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9886298-5a2d-423e-acb6-da7970089c63 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:17.038914 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:17.055621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e9886298-5a2d-423e-acb6-da7970089c63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fed44db7dc014719b5ed0399c715435b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:17.056196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e9886298-5a2d-423e-acb6-da7970089c63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fed44db7dc014719b5ed0399c715435b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:17.069301 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e9886298-5a2d-423e-acb6-da7970089c63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fed44db7dc014719b5ed0399c715435b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:17.076519 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e9886298-5a2d-423e-acb6-da7970089c63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fed44db7dc014719b5ed0399c715435b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:17.083532 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e9886298-5a2d-423e-acb6-da7970089c63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fed44db7dc014719b5ed0399c715435b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:17.090144 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1022/2057] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:16 2022] DELETE /identity/v3/users/fed44db7dc014719b5ed0399c715435b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:17.267660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badc74f-4e5f-42d1-b8dc-d6c4c00fe79b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:17.279429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9badc74f-4e5f-42d1-b8dc-d6c4c00fe79b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:17.343255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211864-ed89-4383-9025-3b78031bfb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:17.344114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211864-ed89-4383-9025-3b78031bfb94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:17.344394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211864-ed89-4383-9025-3b78031bfb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:17.344713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211864-ed89-4383-9025-3b78031bfb94 admin admin] PATH_INFO: `/v3/projects/7a5b4dfcb90d421192be6dbc102ca1bb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:17.345914 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df211864-ed89-4383-9025-3b78031bfb94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a5b4dfcb90d421192be6dbc102ca1bb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:17.347857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df211864-ed89-4383-9025-3b78031bfb94 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:17.350716 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:17.398358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-df211864-ed89-4383-9025-3b78031bfb94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a5b4dfcb90d421192be6dbc102ca1bb', 'request_id': 'req-df211864-ed89-4383-9025-3b78031bfb94'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:17.404771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-df211864-ed89-4383-9025-3b78031bfb94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a5b4dfcb90d421192be6dbc102ca1bb', 'request_id': 'req-df211864-ed89-4383-9025-3b78031bfb94'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:17.410638 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-df211864-ed89-4383-9025-3b78031bfb94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a5b4dfcb90d421192be6dbc102ca1bb', 'request_id': 'req-df211864-ed89-4383-9025-3b78031bfb94'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:17.424816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-df211864-ed89-4383-9025-3b78031bfb94 admin admin] The token cache is being invalidate because project 7a5b4dfcb90d421192be6dbc102ca1bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:17.425148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-df211864-ed89-4383-9025-3b78031bfb94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:17.425768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-df211864-ed89-4383-9025-3b78031bfb94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:17.427712 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1036/2058] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:17 2022] DELETE /identity/v3/projects/7a5b4dfcb90d421192be6dbc102ca1bb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:17.432895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafa571-3514-4f8e-9b15-2c543b7a2acf None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:17.433303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafa571-3514-4f8e-9b15-2c543b7a2acf None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:17.433652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eafa571-3514-4f8e-9b15-2c543b7a2acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:17.453155 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-5eafa571-3514-4f8e-9b15-2c543b7a2acf None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:17.454350 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5eafa571-3514-4f8e-9b15-2c543b7a2acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:17.514175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1023/2059] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:17.522629 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dd1f4-7d5d-40e0-8b3f-d4fea163d9f0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:17.522940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dd1f4-7d5d-40e0-8b3f-d4fea163d9f0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:17.523216 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751dd1f4-7d5d-40e0-8b3f-d4fea163d9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:17.542284 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-751dd1f4-7d5d-40e0-8b3f-d4fea163d9f0 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:17.543488 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-751dd1f4-7d5d-40e0-8b3f-d4fea163d9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:17.551338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1037/2060] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:17.556042 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafa571-3514-4f8e-9b15-2c543b7a2acf None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:17.565672 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eafa571-3514-4f8e-9b15-2c543b7a2acf None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:17.570193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd87c02-d6a0-44d9-953d-90561fad4d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:17.570845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd87c02-d6a0-44d9-953d-90561fad4d16 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:17.571128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd87c02-d6a0-44d9-953d-90561fad4d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:17.571400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd87c02-d6a0-44d9-953d-90561fad4d16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:17.572127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd87c02-d6a0-44d9-953d-90561fad4d16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:15:17.572430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd87c02-d6a0-44d9-953d-90561fad4d16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:17.574044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd87c02-d6a0-44d9-953d-90561fad4d16 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:17.589018 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1024/2061] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:15:17.590008 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dd1f4-7d5d-40e0-8b3f-d4fea163d9f0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:17.599326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751dd1f4-7d5d-40e0-8b3f-d4fea163d9f0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:17.604462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbc277-dca1-40f5-8c4e-2be3456b1ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:17.605144 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbc277-dca1-40f5-8c4e-2be3456b1ba4 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:17.605425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbc277-dca1-40f5-8c4e-2be3456b1ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:17.605698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbc277-dca1-40f5-8c4e-2be3456b1ba4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:17.606392 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbc277-dca1-40f5-8c4e-2be3456b1ba4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:17.608281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fbc277-dca1-40f5-8c4e-2be3456b1ba4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:17.636208 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1038/2062] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:17 2022] POST /identity/v3/projects => generated 405 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:17.637650 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd87c02-d6a0-44d9-953d-90561fad4d16 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:17.648129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd87c02-d6a0-44d9-953d-90561fad4d16 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:17.707043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d61e31-44f1-4724-bed2-e2c30d8c84f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:17.707712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d61e31-44f1-4724-bed2-e2c30d8c84f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:17.707995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d61e31-44f1-4724-bed2-e2c30d8c84f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:17.708272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d61e31-44f1-4724-bed2-e2c30d8c84f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:17.708995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d61e31-44f1-4724-bed2-e2c30d8c84f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:17.710912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d61e31-44f1-4724-bed2-e2c30d8c84f1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:17.730657 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1025/2063] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:17 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:17.734895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbc277-dca1-40f5-8c4e-2be3456b1ba4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:17.746105 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbc277-dca1-40f5-8c4e-2be3456b1ba4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:17.750683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f0027-3bce-4683-ac99-8d1987c7ad32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:17.751334 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f0027-3bce-4683-ac99-8d1987c7ad32 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:17.751620 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f0027-3bce-4683-ac99-8d1987c7ad32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:17.751893 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f0027-3bce-4683-ac99-8d1987c7ad32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:17.752619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f0027-3bce-4683-ac99-8d1987c7ad32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:17.754248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f0027-3bce-4683-ac99-8d1987c7ad32 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:17.764232 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1039/2064] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:17.767957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d61e31-44f1-4724-bed2-e2c30d8c84f1 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:17.777878 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d61e31-44f1-4724-bed2-e2c30d8c84f1 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:17.782655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b414aa2b-d87e-4ff8-9ee8-cf6eab2aa33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:17.783316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414aa2b-d87e-4ff8-9ee8-cf6eab2aa33d admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:17.783601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414aa2b-d87e-4ff8-9ee8-cf6eab2aa33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:17.783874 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b414aa2b-d87e-4ff8-9ee8-cf6eab2aa33d admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:17.784606 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414aa2b-d87e-4ff8-9ee8-cf6eab2aa33d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:17.786336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b414aa2b-d87e-4ff8-9ee8-cf6eab2aa33d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:17.796784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1026/2065] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:17.800785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f0027-3bce-4683-ac99-8d1987c7ad32 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:17.811540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f0027-3bce-4683-ac99-8d1987c7ad32 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:17.816224 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29165eb-09d7-4a83-9530-db0663265cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:17.817067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29165eb-09d7-4a83-9530-db0663265cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:17.817345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29165eb-09d7-4a83-9530-db0663265cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:17.817614 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29165eb-09d7-4a83-9530-db0663265cdd admin admin] PATH_INFO: `/v3/projects/d2b4ebf6afbf466995ef1cd0d6bb46d8/users/ab4786e1b7f54c25a0c495add5ff6883/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:17.831912 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29165eb-09d7-4a83-9530-db0663265cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2b4ebf6afbf466995ef1cd0d6bb46d8, user_id=ab4786e1b7f54c25a0c495add5ff6883, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:17.834909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29165eb-09d7-4a83-9530-db0663265cdd admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:17.848422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1040/2066] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:17 2022] PUT /identity/v3/projects/d2b4ebf6afbf466995ef1cd0d6bb46d8/users/ab4786e1b7f54c25a0c495add5ff6883/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:17.858845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dea510-1fd7-47c8-8732-20c308626c03 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:17.859140 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dea510-1fd7-47c8-8732-20c308626c03 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:17.859420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dea510-1fd7-47c8-8732-20c308626c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:17.882917 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:17.885987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e8dea510-1fd7-47c8-8732-20c308626c03 None None] MFA Rules not processed for user `ab4786e1b7f54c25a0c495add5ff6883`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:17.887932 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e8dea510-1fd7-47c8-8732-20c308626c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:17.954173 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1027/2067] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:17 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:17.958632 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29165eb-09d7-4a83-9530-db0663265cdd admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:18.024375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29165eb-09d7-4a83-9530-db0663265cdd admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:18.029244 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53986684-a998-4295-b2da-7d7b61440b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:18.029913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53986684-a998-4295-b2da-7d7b61440b6b admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:18.030192 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53986684-a998-4295-b2da-7d7b61440b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:18.030466 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53986684-a998-4295-b2da-7d7b61440b6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:18.031170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53986684-a998-4295-b2da-7d7b61440b6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:18.033108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53986684-a998-4295-b2da-7d7b61440b6b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:18.059908 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1041/2068] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:17 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:18.064422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dea510-1fd7-47c8-8732-20c308626c03 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:18.077342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dea510-1fd7-47c8-8732-20c308626c03 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:18.132440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fc0b2-5ad8-471c-a556-8d6271852ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:18.133139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fc0b2-5ad8-471c-a556-8d6271852ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:18.133419 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fc0b2-5ad8-471c-a556-8d6271852ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:18.133691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566fc0b2-5ad8-471c-a556-8d6271852ad1 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:18.134375 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fc0b2-5ad8-471c-a556-8d6271852ad1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:18.136270 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566fc0b2-5ad8-471c-a556-8d6271852ad1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:18.155598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1028/2069] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:18 2022] POST /identity/v3/users => generated 309 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:18.159736 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53986684-a998-4295-b2da-7d7b61440b6b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:18.169862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53986684-a998-4295-b2da-7d7b61440b6b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:18.174439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb7ff6-8982-4bfb-bc9d-4fa7d4a8bd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:18.175091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb7ff6-8982-4bfb-bc9d-4fa7d4a8bd4b admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:18.175371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb7ff6-8982-4bfb-bc9d-4fa7d4a8bd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:18.175648 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bb7ff6-8982-4bfb-bc9d-4fa7d4a8bd4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:18.176375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb7ff6-8982-4bfb-bc9d-4fa7d4a8bd4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:18.178722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bb7ff6-8982-4bfb-bc9d-4fa7d4a8bd4b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:18.188754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1042/2070] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:18.192797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fc0b2-5ad8-471c-a556-8d6271852ad1 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:18.203078 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566fc0b2-5ad8-471c-a556-8d6271852ad1 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:18.207548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2e716-3de2-4b00-bf0d-4fe5485376dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:18.208411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2e716-3de2-4b00-bf0d-4fe5485376dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:18.208687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2e716-3de2-4b00-bf0d-4fe5485376dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:18.208988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc2e716-3de2-4b00-bf0d-4fe5485376dc admin admin] PATH_INFO: `/v3/projects/7a7739dde4c94e368dd17866cf417457/users/8fe9bc2e718c4b6ab6f7bc57b4ac3324/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:18.223040 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2e716-3de2-4b00-bf0d-4fe5485376dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7739dde4c94e368dd17866cf417457, user_id=8fe9bc2e718c4b6ab6f7bc57b4ac3324, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:18.225712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc2e716-3de2-4b00-bf0d-4fe5485376dc admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:18.238400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1029/2071] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:18 2022] PUT /identity/v3/projects/7a7739dde4c94e368dd17866cf417457/users/8fe9bc2e718c4b6ab6f7bc57b4ac3324/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:18.242931 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb7ff6-8982-4bfb-bc9d-4fa7d4a8bd4b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:18.248023 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:18.253966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bb7ff6-8982-4bfb-bc9d-4fa7d4a8bd4b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:18.310795 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644c5132-352f-4177-b77f-af6f45519187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:18.311460 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c5132-352f-4177-b77f-af6f45519187 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:18.311744 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c5132-352f-4177-b77f-af6f45519187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:18.312020 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644c5132-352f-4177-b77f-af6f45519187 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:18.312750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c5132-352f-4177-b77f-af6f45519187 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:18.314377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644c5132-352f-4177-b77f-af6f45519187 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:18.324126 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1043/2072] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:18.328198 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2e716-3de2-4b00-bf0d-4fe5485376dc admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:18.338784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc2e716-3de2-4b00-bf0d-4fe5485376dc admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:18.343399 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e6e95-8544-4f5b-98b4-b1a048d3d068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:18.344279 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e6e95-8544-4f5b-98b4-b1a048d3d068 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:18.344569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e6e95-8544-4f5b-98b4-b1a048d3d068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:18.344844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3e6e95-8544-4f5b-98b4-b1a048d3d068 admin admin] PATH_INFO: `/v3/projects/7a7739dde4c94e368dd17866cf417457/users/8fe9bc2e718c4b6ab6f7bc57b4ac3324/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:18.346860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e6e95-8544-4f5b-98b4-b1a048d3d068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7739dde4c94e368dd17866cf417457, user_id=8fe9bc2e718c4b6ab6f7bc57b4ac3324, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:18.349525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3e6e95-8544-4f5b-98b4-b1a048d3d068 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:18.360439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1030/2073] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:18 2022] PUT /identity/v3/projects/7a7739dde4c94e368dd17866cf417457/users/8fe9bc2e718c4b6ab6f7bc57b4ac3324/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:18.371389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554e2d7-0dca-4ff1-8c05-e9764185edb3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:18.371681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554e2d7-0dca-4ff1-8c05-e9764185edb3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:18.371966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554e2d7-0dca-4ff1-8c05-e9764185edb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:18.397829 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-b554e2d7-0dca-4ff1-8c05-e9764185edb3 None None] MFA Rules not processed for user `8fe9bc2e718c4b6ab6f7bc57b4ac3324`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:18.399091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b554e2d7-0dca-4ff1-8c05-e9764185edb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:18.466269 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1044/2074] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:18 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:18.477913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e6e95-8544-4f5b-98b4-b1a048d3d068 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:18.479591 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8a3e6e95-8544-4f5b-98b4-b1a048d3d068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:18.539644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3e6e95-8544-4f5b-98b4-b1a048d3d068 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:18.543517 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b40dec-1a47-45fb-88c2-22bc3db1b982 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:18.544172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b40dec-1a47-45fb-88c2-22bc3db1b982 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:18.544451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b40dec-1a47-45fb-88c2-22bc3db1b982 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:18.544743 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b40dec-1a47-45fb-88c2-22bc3db1b982 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:18.554425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b40dec-1a47-45fb-88c2-22bc3db1b982 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:18.556427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b40dec-1a47-45fb-88c2-22bc3db1b982 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:18.560490 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:18.569347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1031/2075] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:18 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:18.883560 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554e2d7-0dca-4ff1-8c05-e9764185edb3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:18.948154 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554e2d7-0dca-4ff1-8c05-e9764185edb3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:18.952650 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae9199-d9ff-4f26-987e-66a587891be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:18.953544 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae9199-d9ff-4f26-987e-66a587891be7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:18.953824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae9199-d9ff-4f26-987e-66a587891be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:18.954095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ae9199-d9ff-4f26-987e-66a587891be7 admin admin] PATH_INFO: `/v3/users/ab4786e1b7f54c25a0c495add5ff6883` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:18.955287 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae9199-d9ff-4f26-987e-66a587891be7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab4786e1b7f54c25a0c495add5ff6883)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:18.957639 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ae9199-d9ff-4f26-987e-66a587891be7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:18.987823 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:19.011736 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a4ae9199-d9ff-4f26-987e-66a587891be7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab4786e1b7f54c25a0c495add5ff6883'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:19.012408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a4ae9199-d9ff-4f26-987e-66a587891be7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab4786e1b7f54c25a0c495add5ff6883'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:19.026613 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a4ae9199-d9ff-4f26-987e-66a587891be7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab4786e1b7f54c25a0c495add5ff6883'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:19.034116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a4ae9199-d9ff-4f26-987e-66a587891be7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab4786e1b7f54c25a0c495add5ff6883'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:19.043237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a4ae9199-d9ff-4f26-987e-66a587891be7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab4786e1b7f54c25a0c495add5ff6883'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:19.052279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1045/2076] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:18 2022] DELETE /identity/v3/users/ab4786e1b7f54c25a0c495add5ff6883 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:19.057269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1186e5-01bb-42bc-b0e8-36f667a8bf3d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:19.057731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1186e5-01bb-42bc-b0e8-36f667a8bf3d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:19.057933 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1186e5-01bb-42bc-b0e8-36f667a8bf3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:19.075686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-dd1186e5-01bb-42bc-b0e8-36f667a8bf3d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:19.077019 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-dd1186e5-01bb-42bc-b0e8-36f667a8bf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:19.142510 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1032/2077] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:19.156105 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae9199-d9ff-4f26-987e-66a587891be7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:19.158014 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a4ae9199-d9ff-4f26-987e-66a587891be7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:19.224222 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ae9199-d9ff-4f26-987e-66a587891be7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:19.228873 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3600744-1b0a-48f3-bc56-8fe47ba4c5d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:19.229548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3600744-1b0a-48f3-bc56-8fe47ba4c5d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:19.230035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3600744-1b0a-48f3-bc56-8fe47ba4c5d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:19.230171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3600744-1b0a-48f3-bc56-8fe47ba4c5d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:19.245636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3600744-1b0a-48f3-bc56-8fe47ba4c5d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:19.247284 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3600744-1b0a-48f3-bc56-8fe47ba4c5d6 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:19.259237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1046/2078] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:19.578185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1186e5-01bb-42bc-b0e8-36f667a8bf3d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:19.581212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-dd1186e5-01bb-42bc-b0e8-36f667a8bf3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:19.600548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1186e5-01bb-42bc-b0e8-36f667a8bf3d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:19.607693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f3549-7d93-4f09-9ce4-7c38cf6e9554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:19.608595 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f3549-7d93-4f09-9ce4-7c38cf6e9554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:19.608864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f3549-7d93-4f09-9ce4-7c38cf6e9554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:19.609179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f3549-7d93-4f09-9ce4-7c38cf6e9554 admin admin] PATH_INFO: `/v3/projects/d2b4ebf6afbf466995ef1cd0d6bb46d8` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:19.610990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f3549-7d93-4f09-9ce4-7c38cf6e9554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2b4ebf6afbf466995ef1cd0d6bb46d8)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:19.612991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f3549-7d93-4f09-9ce4-7c38cf6e9554 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:19.701749 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-152f3549-7d93-4f09-9ce4-7c38cf6e9554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2b4ebf6afbf466995ef1cd0d6bb46d8', 'request_id': 'req-152f3549-7d93-4f09-9ce4-7c38cf6e9554'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:19.709427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-152f3549-7d93-4f09-9ce4-7c38cf6e9554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2b4ebf6afbf466995ef1cd0d6bb46d8', 'request_id': 'req-152f3549-7d93-4f09-9ce4-7c38cf6e9554'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:19.720862 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-152f3549-7d93-4f09-9ce4-7c38cf6e9554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2b4ebf6afbf466995ef1cd0d6bb46d8', 'request_id': 'req-152f3549-7d93-4f09-9ce4-7c38cf6e9554'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:19.734688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-152f3549-7d93-4f09-9ce4-7c38cf6e9554 admin admin] The token cache is being invalidate because project d2b4ebf6afbf466995ef1cd0d6bb46d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:19.734979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-152f3549-7d93-4f09-9ce4-7c38cf6e9554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:19.737847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-152f3549-7d93-4f09-9ce4-7c38cf6e9554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:19.743341 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1033/2079] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:19 2022] DELETE /identity/v3/projects/d2b4ebf6afbf466995ef1cd0d6bb46d8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:19.745864 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3600744-1b0a-48f3-bc56-8fe47ba4c5d6 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:19.755806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3600744-1b0a-48f3-bc56-8fe47ba4c5d6 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:19.840823 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225e32f9-ea55-4163-a6f1-022d72f302e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:19.841781 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e32f9-ea55-4163-a6f1-022d72f302e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:19.842019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e32f9-ea55-4163-a6f1-022d72f302e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:19.842290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225e32f9-ea55-4163-a6f1-022d72f302e6 admin admin] PATH_INFO: `/v3/users/8fe9bc2e718c4b6ab6f7bc57b4ac3324` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:19.844821 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e32f9-ea55-4163-a6f1-022d72f302e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fe9bc2e718c4b6ab6f7bc57b4ac3324)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:19.848263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225e32f9-ea55-4163-a6f1-022d72f302e6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:19.913774 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-225e32f9-ea55-4163-a6f1-022d72f302e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fe9bc2e718c4b6ab6f7bc57b4ac3324'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:19.914779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-225e32f9-ea55-4163-a6f1-022d72f302e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fe9bc2e718c4b6ab6f7bc57b4ac3324'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:19.916225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:19.935098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-225e32f9-ea55-4163-a6f1-022d72f302e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fe9bc2e718c4b6ab6f7bc57b4ac3324'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:19.943392 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-225e32f9-ea55-4163-a6f1-022d72f302e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fe9bc2e718c4b6ab6f7bc57b4ac3324'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:19.975467 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-225e32f9-ea55-4163-a6f1-022d72f302e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fe9bc2e718c4b6ab6f7bc57b4ac3324'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:19.982372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1047/2080] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:19 2022] DELETE /identity/v3/users/8fe9bc2e718c4b6ab6f7bc57b4ac3324 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:20.433490 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f3549-7d93-4f09-9ce4-7c38cf6e9554 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:20.434920 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-152f3549-7d93-4f09-9ce4-7c38cf6e9554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:20.448983 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f3549-7d93-4f09-9ce4-7c38cf6e9554 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:20.519356 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d9089-af17-46fd-b242-0446c95524cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:20.520228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d9089-af17-46fd-b242-0446c95524cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:20.520511 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d9089-af17-46fd-b242-0446c95524cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:20.520788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d9089-af17-46fd-b242-0446c95524cd admin admin] PATH_INFO: `/v3/projects/7a7739dde4c94e368dd17866cf417457` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:20.522584 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d9089-af17-46fd-b242-0446c95524cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a7739dde4c94e368dd17866cf417457)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:20.524504 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7d9089-af17-46fd-b242-0446c95524cd admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:20.551784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:20.609095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5e7d9089-af17-46fd-b242-0446c95524cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a7739dde4c94e368dd17866cf417457', 'request_id': 'req-5e7d9089-af17-46fd-b242-0446c95524cd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:20.617105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5e7d9089-af17-46fd-b242-0446c95524cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a7739dde4c94e368dd17866cf417457', 'request_id': 'req-5e7d9089-af17-46fd-b242-0446c95524cd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:20.627813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5e7d9089-af17-46fd-b242-0446c95524cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a7739dde4c94e368dd17866cf417457', 'request_id': 'req-5e7d9089-af17-46fd-b242-0446c95524cd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:20.644888 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5e7d9089-af17-46fd-b242-0446c95524cd admin admin] The token cache is being invalidate because project 7a7739dde4c94e368dd17866cf417457 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:20.645245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5e7d9089-af17-46fd-b242-0446c95524cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:20.645884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5e7d9089-af17-46fd-b242-0446c95524cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:20.648038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1034/2081] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:20 2022] DELETE /identity/v3/projects/7a7739dde4c94e368dd17866cf417457 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:20.653429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6feb78-b81e-4572-a396-b240c67f9ea3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:20.653721 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6feb78-b81e-4572-a396-b240c67f9ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:20.654028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6feb78-b81e-4572-a396-b240c67f9ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:20.673425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1c6feb78-b81e-4572-a396-b240c67f9ea3 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:20.674725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1c6feb78-b81e-4572-a396-b240c67f9ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:20.738983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1048/2082] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:20.748286 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d52cf-39bf-4423-92c1-a6ac6e39d877 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:20.748579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d52cf-39bf-4423-92c1-a6ac6e39d877 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:20.748856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d52cf-39bf-4423-92c1-a6ac6e39d877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:20.768591 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9f5d52cf-39bf-4423-92c1-a6ac6e39d877 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:20.769845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9f5d52cf-39bf-4423-92c1-a6ac6e39d877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:20.778564 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1035/2083] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:20.782790 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6feb78-b81e-4572-a396-b240c67f9ea3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:20.793097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6feb78-b81e-4572-a396-b240c67f9ea3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:20.797745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2f77f-73a7-47b2-9212-b4f766f894af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:20.798406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2f77f-73a7-47b2-9212-b4f766f894af admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:20.798689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2f77f-73a7-47b2-9212-b4f766f894af admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:20.798988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2f77f-73a7-47b2-9212-b4f766f894af admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:20.799790 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2f77f-73a7-47b2-9212-b4f766f894af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:15:20.800094 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2f77f-73a7-47b2-9212-b4f766f894af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:20.801738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2f77f-73a7-47b2-9212-b4f766f894af admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:20.814242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1049/2084] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:15:20.818670 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d52cf-39bf-4423-92c1-a6ac6e39d877 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:20.827946 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d52cf-39bf-4423-92c1-a6ac6e39d877 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:20.832912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251d80f-c0bc-4f8e-8c4e-24f7e6f82b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:20.833598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251d80f-c0bc-4f8e-8c4e-24f7e6f82b1c admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:20.833883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251d80f-c0bc-4f8e-8c4e-24f7e6f82b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:20.834186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251d80f-c0bc-4f8e-8c4e-24f7e6f82b1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:20.835046 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251d80f-c0bc-4f8e-8c4e-24f7e6f82b1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:20.837055 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251d80f-c0bc-4f8e-8c4e-24f7e6f82b1c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:20.870946 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1036/2085] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:20 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:20.875051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2f77f-73a7-47b2-9212-b4f766f894af admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:20.886011 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2f77f-73a7-47b2-9212-b4f766f894af admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:20.946086 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b1c09-8599-401f-adb1-56a4019bb27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:20.946687 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b1c09-8599-401f-adb1-56a4019bb27f admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:20.946971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b1c09-8599-401f-adb1-56a4019bb27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:20.947244 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b1c09-8599-401f-adb1-56a4019bb27f admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:20.947947 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b1c09-8599-401f-adb1-56a4019bb27f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:20.949907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b1c09-8599-401f-adb1-56a4019bb27f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:20.975308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1050/2086] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:20 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:15:20.976307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251d80f-c0bc-4f8e-8c4e-24f7e6f82b1c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:20.988322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251d80f-c0bc-4f8e-8c4e-24f7e6f82b1c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:20.994899 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70071b1e-9304-45c1-b4e2-7e75a4d3dfaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:20.996175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70071b1e-9304-45c1-b4e2-7e75a4d3dfaf admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:20.996496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70071b1e-9304-45c1-b4e2-7e75a4d3dfaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:20.996745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70071b1e-9304-45c1-b4e2-7e75a4d3dfaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:20.997705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70071b1e-9304-45c1-b4e2-7e75a4d3dfaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:20.999352 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70071b1e-9304-45c1-b4e2-7e75a4d3dfaf admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:21.012033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1037/2087] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:21.016003 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b1c09-8599-401f-adb1-56a4019bb27f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:21.026813 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b1c09-8599-401f-adb1-56a4019bb27f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:21.031832 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d572e293-e486-4fd1-97f7-2d75fe8545e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:21.032509 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572e293-e486-4fd1-97f7-2d75fe8545e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:21.032789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572e293-e486-4fd1-97f7-2d75fe8545e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:21.033096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d572e293-e486-4fd1-97f7-2d75fe8545e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:21.033842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572e293-e486-4fd1-97f7-2d75fe8545e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:21.035919 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d572e293-e486-4fd1-97f7-2d75fe8545e2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:21.046248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1051/2088] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:21.050556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70071b1e-9304-45c1-b4e2-7e75a4d3dfaf admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:21.061621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70071b1e-9304-45c1-b4e2-7e75a4d3dfaf admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:21.067570 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72543e57-f271-4c32-a9c2-5492cd1fcb28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:21.068617 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72543e57-f271-4c32-a9c2-5492cd1fcb28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:21.068900 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72543e57-f271-4c32-a9c2-5492cd1fcb28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:21.069283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72543e57-f271-4c32-a9c2-5492cd1fcb28 admin admin] PATH_INFO: `/v3/projects/d84bdd46855d48fa9ff16513259028b2/users/72731b69a221495eaa5f9671d96b1a10/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:21.085472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72543e57-f271-4c32-a9c2-5492cd1fcb28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d84bdd46855d48fa9ff16513259028b2, user_id=72731b69a221495eaa5f9671d96b1a10, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:21.088149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72543e57-f271-4c32-a9c2-5492cd1fcb28 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:21.103342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1038/2089] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:21 2022] PUT /identity/v3/projects/d84bdd46855d48fa9ff16513259028b2/users/72731b69a221495eaa5f9671d96b1a10/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:21.115683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27015fef-8bc5-4f19-9dc4-948b73410486 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:21.115991 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27015fef-8bc5-4f19-9dc4-948b73410486 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:21.116271 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27015fef-8bc5-4f19-9dc4-948b73410486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:21.154012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-27015fef-8bc5-4f19-9dc4-948b73410486 None None] MFA Rules not processed for user `72731b69a221495eaa5f9671d96b1a10`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:21.155548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-27015fef-8bc5-4f19-9dc4-948b73410486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:21.177613 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:21.226628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1052/2090] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:21.230952 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72543e57-f271-4c32-a9c2-5492cd1fcb28 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:21.304530 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72543e57-f271-4c32-a9c2-5492cd1fcb28 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:21.309647 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdf22e-ed53-47dd-b655-9c1577cc25cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:21.310302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdf22e-ed53-47dd-b655-9c1577cc25cf admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:21.310586 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdf22e-ed53-47dd-b655-9c1577cc25cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:21.310859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdf22e-ed53-47dd-b655-9c1577cc25cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:21.311551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdf22e-ed53-47dd-b655-9c1577cc25cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:21.313481 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdf22e-ed53-47dd-b655-9c1577cc25cf admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:21.340736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1039/2091] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:21 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:21.344976 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27015fef-8bc5-4f19-9dc4-948b73410486 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:21.358687 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27015fef-8bc5-4f19-9dc4-948b73410486 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:21.417686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c76d9-a7a7-4a57-b5bc-9380eacc99d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:21.418364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c76d9-a7a7-4a57-b5bc-9380eacc99d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:21.418648 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c76d9-a7a7-4a57-b5bc-9380eacc99d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:21.418920 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c76d9-a7a7-4a57-b5bc-9380eacc99d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:21.419610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c76d9-a7a7-4a57-b5bc-9380eacc99d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:21.421551 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9c76d9-a7a7-4a57-b5bc-9380eacc99d4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:21.441974 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1053/2092] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:21 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:21.446082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdf22e-ed53-47dd-b655-9c1577cc25cf admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:21.457434 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdf22e-ed53-47dd-b655-9c1577cc25cf admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:21.462064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d689d4-2e2d-4ef9-a4d7-7da205facb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:21.462727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d689d4-2e2d-4ef9-a4d7-7da205facb0a admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:21.463037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d689d4-2e2d-4ef9-a4d7-7da205facb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:21.463317 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d689d4-2e2d-4ef9-a4d7-7da205facb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:21.464081 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d689d4-2e2d-4ef9-a4d7-7da205facb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:21.465727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d689d4-2e2d-4ef9-a4d7-7da205facb0a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:21.477154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1040/2093] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:21.481091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c76d9-a7a7-4a57-b5bc-9380eacc99d4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:21.492028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c76d9-a7a7-4a57-b5bc-9380eacc99d4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:21.496874 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784587a8-198c-47e1-8dbc-c71ae22bc776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:21.497563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784587a8-198c-47e1-8dbc-c71ae22bc776 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:21.497851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784587a8-198c-47e1-8dbc-c71ae22bc776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:21.498124 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784587a8-198c-47e1-8dbc-c71ae22bc776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:21.498851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784587a8-198c-47e1-8dbc-c71ae22bc776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:21.500479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784587a8-198c-47e1-8dbc-c71ae22bc776 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:21.511503 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1054/2094] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:21.515704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d689d4-2e2d-4ef9-a4d7-7da205facb0a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:21.527523 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d689d4-2e2d-4ef9-a4d7-7da205facb0a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:21.532273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617cad0-0a7a-44de-ae92-7bbfe4b2deba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:21.533178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617cad0-0a7a-44de-ae92-7bbfe4b2deba admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:21.533515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617cad0-0a7a-44de-ae92-7bbfe4b2deba admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:21.533822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4617cad0-0a7a-44de-ae92-7bbfe4b2deba admin admin] PATH_INFO: `/v3/projects/da8039696c8849c2aef45dc909fdd475/users/9dc75e7c4ccb44c29bb88f18bebebae7/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:21.548845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617cad0-0a7a-44de-ae92-7bbfe4b2deba admin admin] RBAC: Authorizing `identity:create_grant(project_id=da8039696c8849c2aef45dc909fdd475, user_id=9dc75e7c4ccb44c29bb88f18bebebae7, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:21.551600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4617cad0-0a7a-44de-ae92-7bbfe4b2deba admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:21.559118 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=6,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:21.566144 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1041/2095] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:21 2022] PUT /identity/v3/projects/da8039696c8849c2aef45dc909fdd475/users/9dc75e7c4ccb44c29bb88f18bebebae7/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:21.578068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d075ec0-ab3f-4264-8248-2511af2b88cc None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:21.578355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d075ec0-ab3f-4264-8248-2511af2b88cc None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:21.578628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d075ec0-ab3f-4264-8248-2511af2b88cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:21.605511 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2d075ec0-ab3f-4264-8248-2511af2b88cc None None] MFA Rules not processed for user `9dc75e7c4ccb44c29bb88f18bebebae7`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:21.606821 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2d075ec0-ab3f-4264-8248-2511af2b88cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:21.677386 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1055/2096] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:15:21.679126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617cad0-0a7a-44de-ae92-7bbfe4b2deba admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:21.741540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4617cad0-0a7a-44de-ae92-7bbfe4b2deba admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:21.746368 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7d38b-1e6e-4c04-8b19-6462b12bc1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:21.747055 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7d38b-1e6e-4c04-8b19-6462b12bc1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:21.747358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7d38b-1e6e-4c04-8b19-6462b12bc1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:21.747636 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee7d38b-1e6e-4c04-8b19-6462b12bc1d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:21.748334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7d38b-1e6e-4c04-8b19-6462b12bc1d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:21.750261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee7d38b-1e6e-4c04-8b19-6462b12bc1d0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:21.779832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1042/2097] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:21 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:21.781022 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d075ec0-ab3f-4264-8248-2511af2b88cc None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:21.791947 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d075ec0-ab3f-4264-8248-2511af2b88cc None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:21.819119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:21.851587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e40b5-3835-466e-ac81-708984eed21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:21.852292 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e40b5-3835-466e-ac81-708984eed21d admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:21.852566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e40b5-3835-466e-ac81-708984eed21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:21.852833 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099e40b5-3835-466e-ac81-708984eed21d admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:21.853562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e40b5-3835-466e-ac81-708984eed21d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:21.855460 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099e40b5-3835-466e-ac81-708984eed21d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:21.878331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1056/2098] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:21 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:21.879557 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7d38b-1e6e-4c04-8b19-6462b12bc1d0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:21.889895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee7d38b-1e6e-4c04-8b19-6462b12bc1d0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:21.894525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbbc83-4bdf-461f-aa19-473cff2ecc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:21.895176 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbbc83-4bdf-461f-aa19-473cff2ecc81 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:21.895455 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbbc83-4bdf-461f-aa19-473cff2ecc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:21.895726 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fbbc83-4bdf-461f-aa19-473cff2ecc81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:21.896473 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbbc83-4bdf-461f-aa19-473cff2ecc81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:21.898600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fbbc83-4bdf-461f-aa19-473cff2ecc81 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:21.909288 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1043/2099] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:21.913903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e40b5-3835-466e-ac81-708984eed21d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:21.923803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099e40b5-3835-466e-ac81-708984eed21d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:21.928497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cbbef-2f5a-4102-ad93-28d8b27a3719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:21.929415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cbbef-2f5a-4102-ad93-28d8b27a3719 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:21.929700 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cbbef-2f5a-4102-ad93-28d8b27a3719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:21.929998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609cbbef-2f5a-4102-ad93-28d8b27a3719 admin admin] PATH_INFO: `/v3/projects/23d6587610884bffaa0cb407a163df39/users/ac3b65e5cbec437ba5e55a369eabc714/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:21.946410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609cbbef-2f5a-4102-ad93-28d8b27a3719 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d6587610884bffaa0cb407a163df39, user_id=ac3b65e5cbec437ba5e55a369eabc714, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:21.949408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609cbbef-2f5a-4102-ad93-28d8b27a3719 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:21.964344 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1057/2100] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:21 2022] PUT /identity/v3/projects/23d6587610884bffaa0cb407a163df39/users/ac3b65e5cbec437ba5e55a369eabc714/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:21.968548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbbc83-4bdf-461f-aa19-473cff2ecc81 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:21.979414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fbbc83-4bdf-461f-aa19-473cff2ecc81 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:22.034728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6bced-4c56-44bf-8f40-4c369f04a8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:22.035306 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6bced-4c56-44bf-8f40-4c369f04a8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:22.035584 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6bced-4c56-44bf-8f40-4c369f04a8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:22.035851 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6bced-4c56-44bf-8f40-4c369f04a8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:22.036567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6bced-4c56-44bf-8f40-4c369f04a8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:22.038199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6bced-4c56-44bf-8f40-4c369f04a8a8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:22.047945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1044/2101] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:22.052452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cbbef-2f5a-4102-ad93-28d8b27a3719 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:22.062593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609cbbef-2f5a-4102-ad93-28d8b27a3719 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:22.067899 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73098e27-0e9d-43bd-8d2f-7f14dd94ab6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:22.069364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73098e27-0e9d-43bd-8d2f-7f14dd94ab6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:22.069639 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73098e27-0e9d-43bd-8d2f-7f14dd94ab6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:22.069906 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73098e27-0e9d-43bd-8d2f-7f14dd94ab6c admin admin] PATH_INFO: `/v3/projects/23d6587610884bffaa0cb407a163df39/users/ac3b65e5cbec437ba5e55a369eabc714/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:22.071903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73098e27-0e9d-43bd-8d2f-7f14dd94ab6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=23d6587610884bffaa0cb407a163df39, user_id=ac3b65e5cbec437ba5e55a369eabc714, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:22.074567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73098e27-0e9d-43bd-8d2f-7f14dd94ab6c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:22.085862 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1058/2102] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:22 2022] PUT /identity/v3/projects/23d6587610884bffaa0cb407a163df39/users/ac3b65e5cbec437ba5e55a369eabc714/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:22.095995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d19f4-42e0-4bba-ab63-33d21c2bf829 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:22.096285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d19f4-42e0-4bba-ab63-33d21c2bf829 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:22.096559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483d19f4-42e0-4bba-ab63-33d21c2bf829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:22.121863 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-483d19f4-42e0-4bba-ab63-33d21c2bf829 None None] MFA Rules not processed for user `ac3b65e5cbec437ba5e55a369eabc714`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:22.123045 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-483d19f4-42e0-4bba-ab63-33d21c2bf829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:22.181706 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:22.193083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1045/2103] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:22.205030 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73098e27-0e9d-43bd-8d2f-7f14dd94ab6c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:22.207281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-73098e27-0e9d-43bd-8d2f-7f14dd94ab6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:22.280965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73098e27-0e9d-43bd-8d2f-7f14dd94ab6c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:22.284967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa7d06-6341-4004-9b7f-b37e9101a61e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:22.285682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa7d06-6341-4004-9b7f-b37e9101a61e service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:22.286067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa7d06-6341-4004-9b7f-b37e9101a61e service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:22.286667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fa7d06-6341-4004-9b7f-b37e9101a61e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:22.297036 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa7d06-6341-4004-9b7f-b37e9101a61e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:22.298549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fa7d06-6341-4004-9b7f-b37e9101a61e service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:22.310633 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1059/2104] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:22.839489 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d19f4-42e0-4bba-ab63-33d21c2bf829 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:22.841203 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-483d19f4-42e0-4bba-ab63-33d21c2bf829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:22.927203 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa7d06-6341-4004-9b7f-b37e9101a61e service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:22.927716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483d19f4-42e0-4bba-ab63-33d21c2bf829 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:22.933844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df97e7-9321-4f8b-97c6-67cbbc9beae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:22.935297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df97e7-9321-4f8b-97c6-67cbbc9beae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:22.935632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df97e7-9321-4f8b-97c6-67cbbc9beae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:22.935912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df97e7-9321-4f8b-97c6-67cbbc9beae3 admin admin] PATH_INFO: `/v3/users/ec2462d8ae1544a1a225c5599864fb41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:22.937224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df97e7-9321-4f8b-97c6-67cbbc9beae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec2462d8ae1544a1a225c5599864fb41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:22.937824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fa7d06-6341-4004-9b7f-b37e9101a61e service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:22.939505 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df97e7-9321-4f8b-97c6-67cbbc9beae3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:22.941871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539418ce-d148-4c82-8c6e-18ad19110ef6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:22.942600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539418ce-d148-4c82-8c6e-18ad19110ef6 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:22.942954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539418ce-d148-4c82-8c6e-18ad19110ef6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:22.943544 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539418ce-d148-4c82-8c6e-18ad19110ef6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:23.007993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539418ce-d148-4c82-8c6e-18ad19110ef6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:23.010247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539418ce-d148-4c82-8c6e-18ad19110ef6 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:23.019958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1060/2105] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:15:23.034637 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-56df97e7-9321-4f8b-97c6-67cbbc9beae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec2462d8ae1544a1a225c5599864fb41'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:23.035847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-56df97e7-9321-4f8b-97c6-67cbbc9beae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec2462d8ae1544a1a225c5599864fb41'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:23.050291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-56df97e7-9321-4f8b-97c6-67cbbc9beae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec2462d8ae1544a1a225c5599864fb41'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:23.058597 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-56df97e7-9321-4f8b-97c6-67cbbc9beae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec2462d8ae1544a1a225c5599864fb41'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:23.068245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-56df97e7-9321-4f8b-97c6-67cbbc9beae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec2462d8ae1544a1a225c5599864fb41'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:23.077255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1046/2106] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:22 2022] DELETE /identity/v3/users/ec2462d8ae1544a1a225c5599864fb41 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:23.081740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae641ae-67eb-4ccb-84c3-9db43aff9169 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:23.082029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae641ae-67eb-4ccb-84c3-9db43aff9169 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:23.082306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae641ae-67eb-4ccb-84c3-9db43aff9169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:23.105230 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4ae641ae-67eb-4ccb-84c3-9db43aff9169 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:23.107027 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4ae641ae-67eb-4ccb-84c3-9db43aff9169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:23.127050 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:23.175845 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1061/2107] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:15:23.185850 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df97e7-9321-4f8b-97c6-67cbbc9beae3 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:23.187292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-56df97e7-9321-4f8b-97c6-67cbbc9beae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:23.248822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df97e7-9321-4f8b-97c6-67cbbc9beae3 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:23.253418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6b276-0c2b-4eb6-b45c-ff1bf5988ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:23.254138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6b276-0c2b-4eb6-b45c-ff1bf5988ceb service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:23.254422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6b276-0c2b-4eb6-b45c-ff1bf5988ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:23.254719 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be6b276-0c2b-4eb6-b45c-ff1bf5988ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:23.264847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6b276-0c2b-4eb6-b45c-ff1bf5988ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:23.266388 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be6b276-0c2b-4eb6-b45c-ff1bf5988ceb service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:23.278082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1047/2108] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:15:23.561461 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae641ae-67eb-4ccb-84c3-9db43aff9169 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:23.563034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4ae641ae-67eb-4ccb-84c3-9db43aff9169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:23.575808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae641ae-67eb-4ccb-84c3-9db43aff9169 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:23.580470 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed4351-2cc8-4777-8fa9-3b22c167fb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:23.581360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed4351-2cc8-4777-8fa9-3b22c167fb65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:23.581644 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed4351-2cc8-4777-8fa9-3b22c167fb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:23.581916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed4351-2cc8-4777-8fa9-3b22c167fb65 admin admin] PATH_INFO: `/v3/projects/2e6ba76e93d847d78eeacb7888a3fc64` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:23.583126 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed4351-2cc8-4777-8fa9-3b22c167fb65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e6ba76e93d847d78eeacb7888a3fc64)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:23.585159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed4351-2cc8-4777-8fa9-3b22c167fb65 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:23.632319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0bed4351-2cc8-4777-8fa9-3b22c167fb65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e6ba76e93d847d78eeacb7888a3fc64', 'request_id': 'req-0bed4351-2cc8-4777-8fa9-3b22c167fb65'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:23.638448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0bed4351-2cc8-4777-8fa9-3b22c167fb65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e6ba76e93d847d78eeacb7888a3fc64', 'request_id': 'req-0bed4351-2cc8-4777-8fa9-3b22c167fb65'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:23.644144 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0bed4351-2cc8-4777-8fa9-3b22c167fb65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e6ba76e93d847d78eeacb7888a3fc64', 'request_id': 'req-0bed4351-2cc8-4777-8fa9-3b22c167fb65'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:23.656150 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0bed4351-2cc8-4777-8fa9-3b22c167fb65 admin admin] The token cache is being invalidate because project 2e6ba76e93d847d78eeacb7888a3fc64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:23.656439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0bed4351-2cc8-4777-8fa9-3b22c167fb65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:23.657104 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0bed4351-2cc8-4777-8fa9-3b22c167fb65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:23.658823 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1062/2109] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:23 2022] DELETE /identity/v3/projects/2e6ba76e93d847d78eeacb7888a3fc64 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:23.663569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6b276-0c2b-4eb6-b45c-ff1bf5988ceb service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:23.673778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be6b276-0c2b-4eb6-b45c-ff1bf5988ceb service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:23.685334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:23.731629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cafa4f-1f80-4373-b521-07db06c2b91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:23.732520 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cafa4f-1f80-4373-b521-07db06c2b91a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:23.732797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cafa4f-1f80-4373-b521-07db06c2b91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:23.733172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cafa4f-1f80-4373-b521-07db06c2b91a admin admin] PATH_INFO: `/v3/users/23aa82abbe894f37ae99c5070aeec77b` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:23.734360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cafa4f-1f80-4373-b521-07db06c2b91a admin admin] RBAC: Authorizing `identity:delete_user(user_id=23aa82abbe894f37ae99c5070aeec77b)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:23.736295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cafa4f-1f80-4373-b521-07db06c2b91a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:23.782001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a2cafa4f-1f80-4373-b521-07db06c2b91a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23aa82abbe894f37ae99c5070aeec77b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:23.782649 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a2cafa4f-1f80-4373-b521-07db06c2b91a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23aa82abbe894f37ae99c5070aeec77b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:23.796041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a2cafa4f-1f80-4373-b521-07db06c2b91a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23aa82abbe894f37ae99c5070aeec77b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:23.803413 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a2cafa4f-1f80-4373-b521-07db06c2b91a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23aa82abbe894f37ae99c5070aeec77b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:23.813721 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a2cafa4f-1f80-4373-b521-07db06c2b91a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23aa82abbe894f37ae99c5070aeec77b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:23.821227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1048/2110] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:23 2022] DELETE /identity/v3/users/23aa82abbe894f37ae99c5070aeec77b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:24.133433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed4351-2cc8-4777-8fa9-3b22c167fb65 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:24.134989 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0bed4351-2cc8-4777-8fa9-3b22c167fb65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:24.148538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed4351-2cc8-4777-8fa9-3b22c167fb65 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:24.216240 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1973e6-f756-463a-bddd-86c97184bc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:24.217137 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1973e6-f756-463a-bddd-86c97184bc97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:24.217421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1973e6-f756-463a-bddd-86c97184bc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:24.217692 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1973e6-f756-463a-bddd-86c97184bc97 admin admin] PATH_INFO: `/v3/projects/031b08795b8e4375842c78f305700e61` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:24.218892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1973e6-f756-463a-bddd-86c97184bc97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=031b08795b8e4375842c78f305700e61)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:24.220823 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1973e6-f756-463a-bddd-86c97184bc97 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:24.273729 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:24.278092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0b1973e6-f756-463a-bddd-86c97184bc97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '031b08795b8e4375842c78f305700e61', 'request_id': 'req-0b1973e6-f756-463a-bddd-86c97184bc97'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:24.285927 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0b1973e6-f756-463a-bddd-86c97184bc97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '031b08795b8e4375842c78f305700e61', 'request_id': 'req-0b1973e6-f756-463a-bddd-86c97184bc97'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:24.292389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0b1973e6-f756-463a-bddd-86c97184bc97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '031b08795b8e4375842c78f305700e61', 'request_id': 'req-0b1973e6-f756-463a-bddd-86c97184bc97'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:24.306710 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0b1973e6-f756-463a-bddd-86c97184bc97 admin admin] The token cache is being invalidate because project 031b08795b8e4375842c78f305700e61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:24.306985 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0b1973e6-f756-463a-bddd-86c97184bc97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:24.307626 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-0b1973e6-f756-463a-bddd-86c97184bc97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:24.309533 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1063/2111] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:24 2022] DELETE /identity/v3/projects/031b08795b8e4375842c78f305700e61 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:24.314367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f5453-3b74-409c-8b2d-ce6c76338433 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:24.314879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f5453-3b74-409c-8b2d-ce6c76338433 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:24.314879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f5453-3b74-409c-8b2d-ce6c76338433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:24.335159 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4e8f5453-3b74-409c-8b2d-ce6c76338433 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:24.336553 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4e8f5453-3b74-409c-8b2d-ce6c76338433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:24.437754 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1049/2112] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:24.447409 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c68715-04df-45e6-88a9-310016f8a38d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:24.447712 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c68715-04df-45e6-88a9-310016f8a38d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:24.448283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c68715-04df-45e6-88a9-310016f8a38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:24.467249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-17c68715-04df-45e6-88a9-310016f8a38d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:24.472035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-17c68715-04df-45e6-88a9-310016f8a38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:24.482115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1064/2113] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:24.484441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f5453-3b74-409c-8b2d-ce6c76338433 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:24.494705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f5453-3b74-409c-8b2d-ce6c76338433 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:24.499730 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8e33a-5490-4ddf-ade3-ea9bb320eadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:24.500395 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8e33a-5490-4ddf-ade3-ea9bb320eadf admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:24.500672 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8e33a-5490-4ddf-ade3-ea9bb320eadf admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:24.501025 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8e33a-5490-4ddf-ade3-ea9bb320eadf admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:24.501769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8e33a-5490-4ddf-ade3-ea9bb320eadf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:15:24.502074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8e33a-5490-4ddf-ade3-ea9bb320eadf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:24.503669 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8e33a-5490-4ddf-ade3-ea9bb320eadf admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:24.525345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1050/2114] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:15:24.530073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c68715-04df-45e6-88a9-310016f8a38d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:24.542359 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c68715-04df-45e6-88a9-310016f8a38d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:24.550340 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3829937-0f05-48f7-a4e9-b9d07192d6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:24.551038 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3829937-0f05-48f7-a4e9-b9d07192d6a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:24.551487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3829937-0f05-48f7-a4e9-b9d07192d6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:24.551773 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3829937-0f05-48f7-a4e9-b9d07192d6a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:24.554376 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3829937-0f05-48f7-a4e9-b9d07192d6a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:24.557131 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3829937-0f05-48f7-a4e9-b9d07192d6a7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:24.600790 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1065/2115] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:24 2022] POST /identity/v3/projects => generated 387 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:24.604716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8e33a-5490-4ddf-ade3-ea9bb320eadf admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:24.616399 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8e33a-5490-4ddf-ade3-ea9bb320eadf admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:24.638136 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:24.680784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac029b7c-f442-40a5-87f8-096e7a0152b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:24.681496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac029b7c-f442-40a5-87f8-096e7a0152b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:24.681776 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac029b7c-f442-40a5-87f8-096e7a0152b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:24.682072 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac029b7c-f442-40a5-87f8-096e7a0152b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:24.682748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac029b7c-f442-40a5-87f8-096e7a0152b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:24.684659 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac029b7c-f442-40a5-87f8-096e7a0152b6 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:24.708644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1051/2116] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:24 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:24.710217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3829937-0f05-48f7-a4e9-b9d07192d6a7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:24.720726 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3829937-0f05-48f7-a4e9-b9d07192d6a7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:24.725420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96883d0c-e2c5-495f-abfc-8478e44a185b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:24.726084 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96883d0c-e2c5-495f-abfc-8478e44a185b admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:24.726369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96883d0c-e2c5-495f-abfc-8478e44a185b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:24.726640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96883d0c-e2c5-495f-abfc-8478e44a185b admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:24.727373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96883d0c-e2c5-495f-abfc-8478e44a185b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:24.729193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96883d0c-e2c5-495f-abfc-8478e44a185b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:24.739673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1066/2117] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:24.744644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac029b7c-f442-40a5-87f8-096e7a0152b6 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:24.755494 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac029b7c-f442-40a5-87f8-096e7a0152b6 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:24.760444 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cbfedf-059e-4124-881c-5f9a912e7f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:24.761224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbfedf-059e-4124-881c-5f9a912e7f21 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:24.761439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbfedf-059e-4124-881c-5f9a912e7f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:24.761674 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cbfedf-059e-4124-881c-5f9a912e7f21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:24.762395 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cbfedf-059e-4124-881c-5f9a912e7f21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:24.764137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cbfedf-059e-4124-881c-5f9a912e7f21 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:24.777345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1052/2118] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:24.781965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96883d0c-e2c5-495f-abfc-8478e44a185b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:24.795458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96883d0c-e2c5-495f-abfc-8478e44a185b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:24.800321 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88594ae4-2bf0-4793-9356-f32a693dcf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:24.801450 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88594ae4-2bf0-4793-9356-f32a693dcf63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:24.801864 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88594ae4-2bf0-4793-9356-f32a693dcf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:24.802146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88594ae4-2bf0-4793-9356-f32a693dcf63 admin admin] PATH_INFO: `/v3/projects/2a834bf3cf664512823a4ed0c85d17d3/users/f75d15b97cf544849db11e0308cd5ee8/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:24.819558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88594ae4-2bf0-4793-9356-f32a693dcf63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a834bf3cf664512823a4ed0c85d17d3, user_id=f75d15b97cf544849db11e0308cd5ee8, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:24.822623 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88594ae4-2bf0-4793-9356-f32a693dcf63 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:24.838845 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1067/2119] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:24 2022] PUT /identity/v3/projects/2a834bf3cf664512823a4ed0c85d17d3/users/f75d15b97cf544849db11e0308cd5ee8/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:24.850793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cad51-cd13-4f31-9194-de2ebde700d1 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:24.850793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cad51-cd13-4f31-9194-de2ebde700d1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:24.851028 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cad51-cd13-4f31-9194-de2ebde700d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:24.880784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-6f1cad51-cd13-4f31-9194-de2ebde700d1 None None] MFA Rules not processed for user `f75d15b97cf544849db11e0308cd5ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:24.882071 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-6f1cad51-cd13-4f31-9194-de2ebde700d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:24.953881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1053/2120] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:24.958483 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88594ae4-2bf0-4793-9356-f32a693dcf63 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:25.023572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88594ae4-2bf0-4793-9356-f32a693dcf63 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:25.028761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b0a2a-06c8-47a3-a7c5-c4455c237e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:25.029511 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b0a2a-06c8-47a3-a7c5-c4455c237e4a admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:25.029856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b0a2a-06c8-47a3-a7c5-c4455c237e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:25.030124 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b0a2a-06c8-47a3-a7c5-c4455c237e4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:25.030864 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b0a2a-06c8-47a3-a7c5-c4455c237e4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:25.032946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259b0a2a-06c8-47a3-a7c5-c4455c237e4a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:25.059468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1068/2121] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:24 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:25.063642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1cad51-cd13-4f31-9194-de2ebde700d1 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:25.077230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1cad51-cd13-4f31-9194-de2ebde700d1 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:25.140353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03111d3-db9f-4b56-8ee0-11c50267e5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:25.141128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03111d3-db9f-4b56-8ee0-11c50267e5d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:25.141412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03111d3-db9f-4b56-8ee0-11c50267e5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:25.141687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03111d3-db9f-4b56-8ee0-11c50267e5d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:25.142399 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03111d3-db9f-4b56-8ee0-11c50267e5d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:25.144341 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03111d3-db9f-4b56-8ee0-11c50267e5d3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:25.173294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1054/2122] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:25 2022] POST /identity/v3/users => generated 300 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:25.177632 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b0a2a-06c8-47a3-a7c5-c4455c237e4a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:25.188587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b0a2a-06c8-47a3-a7c5-c4455c237e4a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:25.193992 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae234ec1-7046-48df-baf0-e622b798b5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:25.194657 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae234ec1-7046-48df-baf0-e622b798b5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:25.194970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae234ec1-7046-48df-baf0-e622b798b5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:25.195306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae234ec1-7046-48df-baf0-e622b798b5a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:25.196161 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae234ec1-7046-48df-baf0-e622b798b5a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:25.197819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae234ec1-7046-48df-baf0-e622b798b5a9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:25.208888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1069/2123] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:25.212829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03111d3-db9f-4b56-8ee0-11c50267e5d3 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:25.223219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03111d3-db9f-4b56-8ee0-11c50267e5d3 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:25.228479 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a33621-826e-46f2-bd8a-93470dcc53c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:25.229407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a33621-826e-46f2-bd8a-93470dcc53c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:25.229837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a33621-826e-46f2-bd8a-93470dcc53c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:25.230237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a33621-826e-46f2-bd8a-93470dcc53c1 admin admin] PATH_INFO: `/v3/projects/c56587a7578a4d598944211e436ab9f7/users/2136feeddd4e4d3183781a1c9751b69c/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:25.247927 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a33621-826e-46f2-bd8a-93470dcc53c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56587a7578a4d598944211e436ab9f7, user_id=2136feeddd4e4d3183781a1c9751b69c, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:25.250632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a33621-826e-46f2-bd8a-93470dcc53c1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:25.253891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:25.267978 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1055/2124] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:25 2022] PUT /identity/v3/projects/c56587a7578a4d598944211e436ab9f7/users/2136feeddd4e4d3183781a1c9751b69c/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:25.272874 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae234ec1-7046-48df-baf0-e622b798b5a9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:25.285664 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae234ec1-7046-48df-baf0-e622b798b5a9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:25.357325 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:25.379425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602ab67d-42b8-40f3-a45c-9380104da2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:25.380038 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ab67d-42b8-40f3-a45c-9380104da2ad admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:25.380669 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ab67d-42b8-40f3-a45c-9380104da2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:25.380669 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ab67d-42b8-40f3-a45c-9380104da2ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:25.382382 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ab67d-42b8-40f3-a45c-9380104da2ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:25.383996 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602ab67d-42b8-40f3-a45c-9380104da2ad admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:25.395937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1070/2125] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:25.399106 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a33621-826e-46f2-bd8a-93470dcc53c1 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:25.409875 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a33621-826e-46f2-bd8a-93470dcc53c1 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:25.416854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39ec49-2feb-4843-916a-fe02d52e07d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:25.417594 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39ec49-2feb-4843-916a-fe02d52e07d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:25.417876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39ec49-2feb-4843-916a-fe02d52e07d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:25.418153 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39ec49-2feb-4843-916a-fe02d52e07d9 admin admin] PATH_INFO: `/v3/projects/c56587a7578a4d598944211e436ab9f7/users/2136feeddd4e4d3183781a1c9751b69c/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:25.420866 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39ec49-2feb-4843-916a-fe02d52e07d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c56587a7578a4d598944211e436ab9f7, user_id=2136feeddd4e4d3183781a1c9751b69c, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:25.423543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa39ec49-2feb-4843-916a-fe02d52e07d9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:25.436225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1056/2126] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:25 2022] PUT /identity/v3/projects/c56587a7578a4d598944211e436ab9f7/users/2136feeddd4e4d3183781a1c9751b69c/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:25.447622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7570e-6aa0-4ed5-a514-fbcd9969964e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:25.448139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7570e-6aa0-4ed5-a514-fbcd9969964e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:25.448490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7570e-6aa0-4ed5-a514-fbcd9969964e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:25.487109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e5e7570e-6aa0-4ed5-a514-fbcd9969964e None None] MFA Rules not processed for user `2136feeddd4e4d3183781a1c9751b69c`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:25.488827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e5e7570e-6aa0-4ed5-a514-fbcd9969964e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:25.577158 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1071/2127] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:25.588435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39ec49-2feb-4843-916a-fe02d52e07d9 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:25.590443 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fa39ec49-2feb-4843-916a-fe02d52e07d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:25.649121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39ec49-2feb-4843-916a-fe02d52e07d9 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:25.653059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153d4a1-6005-4d1b-9fd8-7428f39a1d63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:25.658609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153d4a1-6005-4d1b-9fd8-7428f39a1d63 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:25.658609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153d4a1-6005-4d1b-9fd8-7428f39a1d63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:25.658609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153d4a1-6005-4d1b-9fd8-7428f39a1d63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:25.666793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153d4a1-6005-4d1b-9fd8-7428f39a1d63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:25.668259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153d4a1-6005-4d1b-9fd8-7428f39a1d63 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:25.678478 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1057/2128] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:25.705742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7570e-6aa0-4ed5-a514-fbcd9969964e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:25.780142 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7570e-6aa0-4ed5-a514-fbcd9969964e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:25.785240 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e619d84-3482-4c6e-be6f-93504e72291e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:25.786108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e619d84-3482-4c6e-be6f-93504e72291e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:25.786592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e619d84-3482-4c6e-be6f-93504e72291e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:25.786944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e619d84-3482-4c6e-be6f-93504e72291e admin admin] PATH_INFO: `/v3/users/f75d15b97cf544849db11e0308cd5ee8` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:25.788756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e619d84-3482-4c6e-be6f-93504e72291e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f75d15b97cf544849db11e0308cd5ee8)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:25.790980 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e619d84-3482-4c6e-be6f-93504e72291e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:25.861285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6e619d84-3482-4c6e-be6f-93504e72291e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f75d15b97cf544849db11e0308cd5ee8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:25.861980 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6e619d84-3482-4c6e-be6f-93504e72291e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f75d15b97cf544849db11e0308cd5ee8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:25.879071 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6e619d84-3482-4c6e-be6f-93504e72291e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f75d15b97cf544849db11e0308cd5ee8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:25.883958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:25.888679 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6e619d84-3482-4c6e-be6f-93504e72291e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f75d15b97cf544849db11e0308cd5ee8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:25.899506 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6e619d84-3482-4c6e-be6f-93504e72291e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f75d15b97cf544849db11e0308cd5ee8'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:25.909583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1072/2129] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:25 2022] DELETE /identity/v3/users/f75d15b97cf544849db11e0308cd5ee8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:25.914795 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8d873-488b-4e60-93cb-fb0066904354 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:25.915398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8d873-488b-4e60-93cb-fb0066904354 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:25.915864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e8d873-488b-4e60-93cb-fb0066904354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:25.961310 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b5e8d873-488b-4e60-93cb-fb0066904354 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:25.963010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b5e8d873-488b-4e60-93cb-fb0066904354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:26.083484 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1058/2130] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:26.094028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e619d84-3482-4c6e-be6f-93504e72291e admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:26.095967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6e619d84-3482-4c6e-be6f-93504e72291e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:26.161729 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e619d84-3482-4c6e-be6f-93504e72291e admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:26.165905 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e7c9f-c60d-48e7-a328-8e234d594dfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:26.166548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e7c9f-c60d-48e7-a328-8e234d594dfe service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:26.166827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e7c9f-c60d-48e7-a328-8e234d594dfe service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:26.167095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e7c9f-c60d-48e7-a328-8e234d594dfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:26.176097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e7c9f-c60d-48e7-a328-8e234d594dfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:26.177548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e7c9f-c60d-48e7-a328-8e234d594dfe service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:26.187260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1073/2131] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:26.494233 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8d873-488b-4e60-93cb-fb0066904354 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:26.495844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b5e8d873-488b-4e60-93cb-fb0066904354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:26.508297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e8d873-488b-4e60-93cb-fb0066904354 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:26.512975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742fdea-5457-4caf-9a59-aeb9527abed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:26.513841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742fdea-5457-4caf-9a59-aeb9527abed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:26.514122 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742fdea-5457-4caf-9a59-aeb9527abed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:26.514394 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742fdea-5457-4caf-9a59-aeb9527abed8 admin admin] PATH_INFO: `/v3/projects/2a834bf3cf664512823a4ed0c85d17d3` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:26.515581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742fdea-5457-4caf-9a59-aeb9527abed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a834bf3cf664512823a4ed0c85d17d3)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:26.517545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742fdea-5457-4caf-9a59-aeb9527abed8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:26.564551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b742fdea-5457-4caf-9a59-aeb9527abed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a834bf3cf664512823a4ed0c85d17d3', 'request_id': 'req-b742fdea-5457-4caf-9a59-aeb9527abed8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:26.571123 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b742fdea-5457-4caf-9a59-aeb9527abed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a834bf3cf664512823a4ed0c85d17d3', 'request_id': 'req-b742fdea-5457-4caf-9a59-aeb9527abed8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:26.578854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b742fdea-5457-4caf-9a59-aeb9527abed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a834bf3cf664512823a4ed0c85d17d3', 'request_id': 'req-b742fdea-5457-4caf-9a59-aeb9527abed8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:26.590490 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b742fdea-5457-4caf-9a59-aeb9527abed8 admin admin] The token cache is being invalidate because project 2a834bf3cf664512823a4ed0c85d17d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:26.590780 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b742fdea-5457-4caf-9a59-aeb9527abed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:26.591413 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b742fdea-5457-4caf-9a59-aeb9527abed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:26.592991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1059/2132] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:26 2022] DELETE /identity/v3/projects/2a834bf3cf664512823a4ed0c85d17d3 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:26.597509 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e7c9f-c60d-48e7-a328-8e234d594dfe service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:26.607740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e7c9f-c60d-48e7-a328-8e234d594dfe service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:26.665618 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006cee0-2dd4-41f6-9cdb-4d709ee73731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:26.666495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006cee0-2dd4-41f6-9cdb-4d709ee73731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:26.666778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006cee0-2dd4-41f6-9cdb-4d709ee73731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:26.667051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006cee0-2dd4-41f6-9cdb-4d709ee73731 admin admin] PATH_INFO: `/v3/users/2136feeddd4e4d3183781a1c9751b69c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:26.668340 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006cee0-2dd4-41f6-9cdb-4d709ee73731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2136feeddd4e4d3183781a1c9751b69c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:26.670232 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006cee0-2dd4-41f6-9cdb-4d709ee73731 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:26.726246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f006cee0-2dd4-41f6-9cdb-4d709ee73731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2136feeddd4e4d3183781a1c9751b69c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:26.727336 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f006cee0-2dd4-41f6-9cdb-4d709ee73731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2136feeddd4e4d3183781a1c9751b69c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:26.746135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f006cee0-2dd4-41f6-9cdb-4d709ee73731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2136feeddd4e4d3183781a1c9751b69c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:26.756979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f006cee0-2dd4-41f6-9cdb-4d709ee73731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2136feeddd4e4d3183781a1c9751b69c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:26.766012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f006cee0-2dd4-41f6-9cdb-4d709ee73731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2136feeddd4e4d3183781a1c9751b69c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:26.775690 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1074/2133] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:26 2022] DELETE /identity/v3/users/2136feeddd4e4d3183781a1c9751b69c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:27.049486 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742fdea-5457-4caf-9a59-aeb9527abed8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:27.051343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b742fdea-5457-4caf-9a59-aeb9527abed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:27.084148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742fdea-5457-4caf-9a59-aeb9527abed8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:27.097391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:27.160621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09c0b9-5912-4b77-85b9-63e0df931812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:27.161758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09c0b9-5912-4b77-85b9-63e0df931812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:27.162407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09c0b9-5912-4b77-85b9-63e0df931812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:27.162992 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09c0b9-5912-4b77-85b9-63e0df931812 admin admin] PATH_INFO: `/v3/projects/c56587a7578a4d598944211e436ab9f7` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:27.164578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09c0b9-5912-4b77-85b9-63e0df931812 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c56587a7578a4d598944211e436ab9f7)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:27.167058 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09c0b9-5912-4b77-85b9-63e0df931812 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:27.226251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7a09c0b9-5912-4b77-85b9-63e0df931812 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c56587a7578a4d598944211e436ab9f7', 'request_id': 'req-7a09c0b9-5912-4b77-85b9-63e0df931812'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:27.244873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7a09c0b9-5912-4b77-85b9-63e0df931812 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c56587a7578a4d598944211e436ab9f7', 'request_id': 'req-7a09c0b9-5912-4b77-85b9-63e0df931812'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:27.257838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7a09c0b9-5912-4b77-85b9-63e0df931812 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c56587a7578a4d598944211e436ab9f7', 'request_id': 'req-7a09c0b9-5912-4b77-85b9-63e0df931812'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:27.270310 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7a09c0b9-5912-4b77-85b9-63e0df931812 admin admin] The token cache is being invalidate because project c56587a7578a4d598944211e436ab9f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:27.270670 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7a09c0b9-5912-4b77-85b9-63e0df931812 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:27.271438 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7a09c0b9-5912-4b77-85b9-63e0df931812 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:27.279294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1060/2134] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:27 2022] DELETE /identity/v3/projects/c56587a7578a4d598944211e436ab9f7 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:27.282930 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20da41-c03d-4196-a50e-b9bafcdcd4a1 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:27.283378 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20da41-c03d-4196-a50e-b9bafcdcd4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:27.283470 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20da41-c03d-4196-a50e-b9bafcdcd4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:27.283612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09c0b9-5912-4b77-85b9-63e0df931812 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:27.285470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7a09c0b9-5912-4b77-85b9-63e0df931812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:27.322374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2d20da41-c03d-4196-a50e-b9bafcdcd4a1 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:27.323603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2d20da41-c03d-4196-a50e-b9bafcdcd4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:27.339728 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:27.443211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09c0b9-5912-4b77-85b9-63e0df931812 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:27.445610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1075/2135] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:27.447323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26da77-8a4a-4651-a9b6-830b3dd39357 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:27.447612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26da77-8a4a-4651-a9b6-830b3dd39357 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:27.447893 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac26da77-8a4a-4651-a9b6-830b3dd39357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:27.459999 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccf610-17d0-429d-a040-e2105c94d657 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:27.460667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccf610-17d0-429d-a040-e2105c94d657 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:27.460989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccf610-17d0-429d-a040-e2105c94d657 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:27.461273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ccf610-17d0-429d-a040-e2105c94d657 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:27.465844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-68ccf610-17d0-429d-a040-e2105c94d657 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:27.468751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ac26da77-8a4a-4651-a9b6-830b3dd39357 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:27.470672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ac26da77-8a4a-4651-a9b6-830b3dd39357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:27.479410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1076/2136] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:27.483391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26da77-8a4a-4651-a9b6-830b3dd39357 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:27.494427 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac26da77-8a4a-4651-a9b6-830b3dd39357 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:27.507043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16e895-9695-4373-b0aa-345f5231cd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:27.507947 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16e895-9695-4373-b0aa-345f5231cd7b admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:27.508179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16e895-9695-4373-b0aa-345f5231cd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:27.508442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac16e895-9695-4373-b0aa-345f5231cd7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:27.510213 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16e895-9695-4373-b0aa-345f5231cd7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:15:27.510332 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16e895-9695-4373-b0aa-345f5231cd7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:27.513896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac16e895-9695-4373-b0aa-345f5231cd7b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:27.542810 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1077/2137] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:15:27.543709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16e895-9695-4373-b0aa-345f5231cd7b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:27.544141 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:27.547923 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ccf610-17d0-429d-a040-e2105c94d657 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:27.550505 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ccf610-17d0-429d-a040-e2105c94d657 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:27.557345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac16e895-9695-4373-b0aa-345f5231cd7b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:27.564475 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a392094-1538-407f-b0b2-5defcd950a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:27.565268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a392094-1538-407f-b0b2-5defcd950a31 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:27.565765 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a392094-1538-407f-b0b2-5defcd950a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:27.566121 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a392094-1538-407f-b0b2-5defcd950a31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:27.566910 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a392094-1538-407f-b0b2-5defcd950a31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:27.569766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a392094-1538-407f-b0b2-5defcd950a31 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:27.574297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1061/2138] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:27.605971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1078/2139] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:27 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:27.610416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccf610-17d0-429d-a040-e2105c94d657 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:27.625141 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ccf610-17d0-429d-a040-e2105c94d657 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:27.693131 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b6dda-2c64-4f8e-9e1b-48fd7b3ba814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:27.694370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b6dda-2c64-4f8e-9e1b-48fd7b3ba814 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:27.694370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b6dda-2c64-4f8e-9e1b-48fd7b3ba814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:27.694583 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b6dda-2c64-4f8e-9e1b-48fd7b3ba814 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:27.695259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b6dda-2c64-4f8e-9e1b-48fd7b3ba814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:27.697706 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869b6dda-2c64-4f8e-9e1b-48fd7b3ba814 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:27.725766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1062/2140] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:27 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:27.727129 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a392094-1538-407f-b0b2-5defcd950a31 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:27.742263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a392094-1538-407f-b0b2-5defcd950a31 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:27.747259 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afd1f0-b934-4dd0-b29b-0aa1b4637288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:27.747924 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afd1f0-b934-4dd0-b29b-0aa1b4637288 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:27.748201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afd1f0-b934-4dd0-b29b-0aa1b4637288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:27.748686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88afd1f0-b934-4dd0-b29b-0aa1b4637288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:27.749412 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afd1f0-b934-4dd0-b29b-0aa1b4637288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:27.751090 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88afd1f0-b934-4dd0-b29b-0aa1b4637288 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:27.763547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1079/2141] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:27.767609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b6dda-2c64-4f8e-9e1b-48fd7b3ba814 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:27.780194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b6dda-2c64-4f8e-9e1b-48fd7b3ba814 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:27.786882 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204e4992-52d7-49cc-b312-755b8fa1ca9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:27.789280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e4992-52d7-49cc-b312-755b8fa1ca9b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:27.789739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e4992-52d7-49cc-b312-755b8fa1ca9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:27.790053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204e4992-52d7-49cc-b312-755b8fa1ca9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:27.790797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e4992-52d7-49cc-b312-755b8fa1ca9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:27.793569 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204e4992-52d7-49cc-b312-755b8fa1ca9b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:27.808968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1063/2142] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:27.810787 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afd1f0-b934-4dd0-b29b-0aa1b4637288 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:27.836464 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88afd1f0-b934-4dd0-b29b-0aa1b4637288 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:27.856395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d160d74-4c9e-4f54-9a01-e96989065962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:27.857337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d160d74-4c9e-4f54-9a01-e96989065962 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:27.857620 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d160d74-4c9e-4f54-9a01-e96989065962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:27.857892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d160d74-4c9e-4f54-9a01-e96989065962 admin admin] PATH_INFO: `/v3/projects/45a2bc23fc1f45f29d2e497fe5dbd482/users/33d0db7375f94cedb7c8f8375c31d93f/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:27.910606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d160d74-4c9e-4f54-9a01-e96989065962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45a2bc23fc1f45f29d2e497fe5dbd482, user_id=33d0db7375f94cedb7c8f8375c31d93f, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:27.914447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d160d74-4c9e-4f54-9a01-e96989065962 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:27.946473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1080/2143] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:27 2022] PUT /identity/v3/projects/45a2bc23fc1f45f29d2e497fe5dbd482/users/33d0db7375f94cedb7c8f8375c31d93f/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:27.957353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f0489-7a03-4859-8945-be9abff9baa6 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:27.958062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f0489-7a03-4859-8945-be9abff9baa6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:27.958552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f0489-7a03-4859-8945-be9abff9baa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:28.009419 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a02f0489-7a03-4859-8945-be9abff9baa6 None None] MFA Rules not processed for user `33d0db7375f94cedb7c8f8375c31d93f`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:28.011951 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a02f0489-7a03-4859-8945-be9abff9baa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:28.132536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1064/2144] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:28.136773 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d160d74-4c9e-4f54-9a01-e96989065962 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:28.211854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d160d74-4c9e-4f54-9a01-e96989065962 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:28.216720 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1f6ba-581f-44c3-86fa-97099ef52b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:28.217417 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1f6ba-581f-44c3-86fa-97099ef52b28 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:28.217717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1f6ba-581f-44c3-86fa-97099ef52b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:28.217999 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1f6ba-581f-44c3-86fa-97099ef52b28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:28.218704 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1f6ba-581f-44c3-86fa-97099ef52b28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:28.220810 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1f6ba-581f-44c3-86fa-97099ef52b28 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:28.262596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1081/2145] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:28 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:28.266519 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f0489-7a03-4859-8945-be9abff9baa6 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:28.278016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f0489-7a03-4859-8945-be9abff9baa6 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:28.346231 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12369657-1933-4eb6-bc52-39cd2715f429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:28.347109 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12369657-1933-4eb6-bc52-39cd2715f429 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:28.347432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12369657-1933-4eb6-bc52-39cd2715f429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:28.347731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12369657-1933-4eb6-bc52-39cd2715f429 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:28.348500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12369657-1933-4eb6-bc52-39cd2715f429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:28.350581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12369657-1933-4eb6-bc52-39cd2715f429 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:28.359275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:28.382095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1065/2146] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:28 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:28.386076 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1f6ba-581f-44c3-86fa-97099ef52b28 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:28.397232 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1f6ba-581f-44c3-86fa-97099ef52b28 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:28.404287 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b651cca-c0b9-48c7-b8d4-6aff198bb33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:28.405209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b651cca-c0b9-48c7-b8d4-6aff198bb33c admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:28.405448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b651cca-c0b9-48c7-b8d4-6aff198bb33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:28.405782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b651cca-c0b9-48c7-b8d4-6aff198bb33c admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:28.406642 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b651cca-c0b9-48c7-b8d4-6aff198bb33c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:28.408629 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b651cca-c0b9-48c7-b8d4-6aff198bb33c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:28.437540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1082/2147] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:15:28.438776 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12369657-1933-4eb6-bc52-39cd2715f429 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:28.455402 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12369657-1933-4eb6-bc52-39cd2715f429 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:28.484006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4f7c4-df1f-43a0-b504-584592bea724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:28.485100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4f7c4-df1f-43a0-b504-584592bea724 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:28.488667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4f7c4-df1f-43a0-b504-584592bea724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:28.489917 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4f7c4-df1f-43a0-b504-584592bea724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:28.492398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4f7c4-df1f-43a0-b504-584592bea724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:28.494613 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4f7c4-df1f-43a0-b504-584592bea724 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:28.520784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1066/2148] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:28.525187 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b651cca-c0b9-48c7-b8d4-6aff198bb33c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:28.538545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b651cca-c0b9-48c7-b8d4-6aff198bb33c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:28.555676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba073de-eb58-4d3c-ade4-103ce37f5bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:28.557379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba073de-eb58-4d3c-ade4-103ce37f5bb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:28.558050 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba073de-eb58-4d3c-ade4-103ce37f5bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:28.558478 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba073de-eb58-4d3c-ade4-103ce37f5bb9 admin admin] PATH_INFO: `/v3/projects/3b3dd50d1e57413b829e8d62ce1049c7/users/4987b5aa0eb44ac5b70633f5fc7b35d1/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:28.562000 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:28.590656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba073de-eb58-4d3c-ade4-103ce37f5bb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b3dd50d1e57413b829e8d62ce1049c7, user_id=4987b5aa0eb44ac5b70633f5fc7b35d1, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:28.594018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba073de-eb58-4d3c-ade4-103ce37f5bb9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:28.622151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1083/2149] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:28 2022] PUT /identity/v3/projects/3b3dd50d1e57413b829e8d62ce1049c7/users/4987b5aa0eb44ac5b70633f5fc7b35d1/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:28.633769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e01f8e-5a36-4e34-ab2b-a9b7b8a5ecfa None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:28.634220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e01f8e-5a36-4e34-ab2b-a9b7b8a5ecfa None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:28.634514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e01f8e-5a36-4e34-ab2b-a9b7b8a5ecfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:28.691018 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-21e01f8e-5a36-4e34-ab2b-a9b7b8a5ecfa None None] MFA Rules not processed for user `4987b5aa0eb44ac5b70633f5fc7b35d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:28.697072 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-21e01f8e-5a36-4e34-ab2b-a9b7b8a5ecfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:28.801615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1067/2150] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:28.807575 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba073de-eb58-4d3c-ade4-103ce37f5bb9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:28.889683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba073de-eb58-4d3c-ade4-103ce37f5bb9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:28.895406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18df90-5cc9-44e3-9982-f31a39e4c3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:28.896179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18df90-5cc9-44e3-9982-f31a39e4c3a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:28.896568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18df90-5cc9-44e3-9982-f31a39e4c3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:28.896986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18df90-5cc9-44e3-9982-f31a39e4c3a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:28.898299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18df90-5cc9-44e3-9982-f31a39e4c3a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:28.899871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18df90-5cc9-44e3-9982-f31a39e4c3a1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:28.932296 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1084/2151] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:28 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:15:28.933324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e01f8e-5a36-4e34-ab2b-a9b7b8a5ecfa None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:28.944494 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e01f8e-5a36-4e34-ab2b-a9b7b8a5ecfa None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:29.005630 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529303c2-9165-463d-83c0-7bf1d338b0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:29.006394 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529303c2-9165-463d-83c0-7bf1d338b0c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:29.006759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529303c2-9165-463d-83c0-7bf1d338b0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:29.007048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529303c2-9165-463d-83c0-7bf1d338b0c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:29.007824 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529303c2-9165-463d-83c0-7bf1d338b0c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:29.009881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529303c2-9165-463d-83c0-7bf1d338b0c3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:29.030977 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1068/2152] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:28 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:29.034712 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18df90-5cc9-44e3-9982-f31a39e4c3a1 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:29.045828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18df90-5cc9-44e3-9982-f31a39e4c3a1 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:29.051719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd14654-eeaa-4e88-a61f-465307bfee09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:29.052508 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd14654-eeaa-4e88-a61f-465307bfee09 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:29.053510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd14654-eeaa-4e88-a61f-465307bfee09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:29.053783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd14654-eeaa-4e88-a61f-465307bfee09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:29.054583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd14654-eeaa-4e88-a61f-465307bfee09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:29.056264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd14654-eeaa-4e88-a61f-465307bfee09 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:29.071908 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1085/2153] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:15:29.075818 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529303c2-9165-463d-83c0-7bf1d338b0c3 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:29.087863 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529303c2-9165-463d-83c0-7bf1d338b0c3 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:29.093274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ad75b-1891-4c27-aedb-a94eb99a8da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:29.094624 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ad75b-1891-4c27-aedb-a94eb99a8da2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:29.095112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ad75b-1891-4c27-aedb-a94eb99a8da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:29.095561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36ad75b-1891-4c27-aedb-a94eb99a8da2 admin admin] PATH_INFO: `/v3/projects/f68a3a7c657646f6b46d382f052469b9/users/f9fe5e98b83244379452ad5d312c06ad/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:29.113029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ad75b-1891-4c27-aedb-a94eb99a8da2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f68a3a7c657646f6b46d382f052469b9, user_id=f9fe5e98b83244379452ad5d312c06ad, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:29.116359 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36ad75b-1891-4c27-aedb-a94eb99a8da2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:29.135955 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1069/2154] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:29 2022] PUT /identity/v3/projects/f68a3a7c657646f6b46d382f052469b9/users/f9fe5e98b83244379452ad5d312c06ad/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:29.137129 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd14654-eeaa-4e88-a61f-465307bfee09 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:29.148290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd14654-eeaa-4e88-a61f-465307bfee09 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:29.222199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5184143-3fd1-49ba-87f6-e819f3f8d2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:29.222879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5184143-3fd1-49ba-87f6-e819f3f8d2f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:29.223156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5184143-3fd1-49ba-87f6-e819f3f8d2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:29.223424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5184143-3fd1-49ba-87f6-e819f3f8d2f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:29.224160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5184143-3fd1-49ba-87f6-e819f3f8d2f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:29.226388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5184143-3fd1-49ba-87f6-e819f3f8d2f3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:29.245604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1086/2155] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:29.246991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ad75b-1891-4c27-aedb-a94eb99a8da2 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:29.268738 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36ad75b-1891-4c27-aedb-a94eb99a8da2 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:29.275319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f41d2-ce36-403d-b39a-124c8e290b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:29.276369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f41d2-ce36-403d-b39a-124c8e290b0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:29.276751 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f41d2-ce36-403d-b39a-124c8e290b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:29.277409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f41d2-ce36-403d-b39a-124c8e290b0d admin admin] PATH_INFO: `/v3/projects/f68a3a7c657646f6b46d382f052469b9/users/f9fe5e98b83244379452ad5d312c06ad/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:29.279767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f41d2-ce36-403d-b39a-124c8e290b0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f68a3a7c657646f6b46d382f052469b9, user_id=f9fe5e98b83244379452ad5d312c06ad, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:29.282843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361f41d2-ce36-403d-b39a-124c8e290b0d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:29.306544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1070/2156] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:29 2022] PUT /identity/v3/projects/f68a3a7c657646f6b46d382f052469b9/users/f9fe5e98b83244379452ad5d312c06ad/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:29.308169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3f1e7-685e-43c6-8ed2-9741992fb764 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:29.308650 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3f1e7-685e-43c6-8ed2-9741992fb764 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:29.309168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3f1e7-685e-43c6-8ed2-9741992fb764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:29.387226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-92a3f1e7-685e-43c6-8ed2-9741992fb764 None None] MFA Rules not processed for user `f9fe5e98b83244379452ad5d312c06ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:29.388741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-92a3f1e7-685e-43c6-8ed2-9741992fb764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:29.486757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:29.535358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1087/2157] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:29 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:15:29.548839 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f41d2-ce36-403d-b39a-124c8e290b0d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:29.551493 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-361f41d2-ce36-403d-b39a-124c8e290b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:29.586853 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:29.661254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361f41d2-ce36-403d-b39a-124c8e290b0d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:29.667672 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe926c-6f5d-4983-a92b-73ca59102fb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:29.668713 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe926c-6f5d-4983-a92b-73ca59102fb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:29.669176 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe926c-6f5d-4983-a92b-73ca59102fb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:29.669745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe926c-6f5d-4983-a92b-73ca59102fb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:29.684716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe926c-6f5d-4983-a92b-73ca59102fb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:29.686600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe926c-6f5d-4983-a92b-73ca59102fb4 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:29.701442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1071/2158] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:29 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:29.767098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3f1e7-685e-43c6-8ed2-9741992fb764 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:29.781888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3f1e7-685e-43c6-8ed2-9741992fb764 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:29.790778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e752d8c4-78a5-4ab2-83ed-42e608e09868 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:29.791281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752d8c4-78a5-4ab2-83ed-42e608e09868 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:29.791609 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752d8c4-78a5-4ab2-83ed-42e608e09868 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:29.791887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e752d8c4-78a5-4ab2-83ed-42e608e09868 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:29.888055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752d8c4-78a5-4ab2-83ed-42e608e09868 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:29.889715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e752d8c4-78a5-4ab2-83ed-42e608e09868 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:29.900419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1088/2159] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:30.419433 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe926c-6f5d-4983-a92b-73ca59102fb4 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:30.420845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f3fe926c-6f5d-4983-a92b-73ca59102fb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:30.503145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe926c-6f5d-4983-a92b-73ca59102fb4 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:30.508742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ae6ef-800c-436b-9047-778b37d91446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:30.509731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ae6ef-800c-436b-9047-778b37d91446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:30.510017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ae6ef-800c-436b-9047-778b37d91446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:30.510290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ae6ef-800c-436b-9047-778b37d91446 admin admin] PATH_INFO: `/v3/users/72731b69a221495eaa5f9671d96b1a10` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:30.511500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ae6ef-800c-436b-9047-778b37d91446 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72731b69a221495eaa5f9671d96b1a10)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:30.513457 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ae6ef-800c-436b-9047-778b37d91446 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:30.562229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ac9ae6ef-800c-436b-9047-778b37d91446 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72731b69a221495eaa5f9671d96b1a10'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:30.562937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ac9ae6ef-800c-436b-9047-778b37d91446 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72731b69a221495eaa5f9671d96b1a10'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:30.575883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ac9ae6ef-800c-436b-9047-778b37d91446 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72731b69a221495eaa5f9671d96b1a10'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:30.582609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ac9ae6ef-800c-436b-9047-778b37d91446 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72731b69a221495eaa5f9671d96b1a10'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:30.590215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ac9ae6ef-800c-436b-9047-778b37d91446 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72731b69a221495eaa5f9671d96b1a10'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:30.596197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1072/2160] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:30 2022] DELETE /identity/v3/users/72731b69a221495eaa5f9671d96b1a10 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:30.601281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec09e6-48e5-4fa0-b9c7-fcb6c3c3e772 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:30.601765 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec09e6-48e5-4fa0-b9c7-fcb6c3c3e772 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:30.601858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec09e6-48e5-4fa0-b9c7-fcb6c3c3e772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:30.619317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-dfec09e6-48e5-4fa0-b9c7-fcb6c3c3e772 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:30.620514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-dfec09e6-48e5-4fa0-b9c7-fcb6c3c3e772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:30.679722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1089/2161] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:30.689260 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ae6ef-800c-436b-9047-778b37d91446 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:30.690633 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ac9ae6ef-800c-436b-9047-778b37d91446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:30.736299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:30.748335 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ae6ef-800c-436b-9047-778b37d91446 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:30.752126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c97b7a-c4ed-45c9-9952-2e0b0dd3c5e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:30.752767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c97b7a-c4ed-45c9-9952-2e0b0dd3c5e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:30.753082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c97b7a-c4ed-45c9-9952-2e0b0dd3c5e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:30.753354 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c97b7a-c4ed-45c9-9952-2e0b0dd3c5e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:30.762754 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c97b7a-c4ed-45c9-9952-2e0b0dd3c5e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:30.764177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c97b7a-c4ed-45c9-9952-2e0b0dd3c5e8 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:30.773442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1073/2162] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:31.107789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec09e6-48e5-4fa0-b9c7-fcb6c3c3e772 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:31.109280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-dfec09e6-48e5-4fa0-b9c7-fcb6c3c3e772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:31.121845 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec09e6-48e5-4fa0-b9c7-fcb6c3c3e772 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:31.126556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475d761-da1f-4b23-91da-79ac5828eef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:31.127900 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475d761-da1f-4b23-91da-79ac5828eef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:31.127900 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475d761-da1f-4b23-91da-79ac5828eef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:31.128115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475d761-da1f-4b23-91da-79ac5828eef6 admin admin] PATH_INFO: `/v3/projects/d84bdd46855d48fa9ff16513259028b2` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:31.129189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475d761-da1f-4b23-91da-79ac5828eef6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d84bdd46855d48fa9ff16513259028b2)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:31.131306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475d761-da1f-4b23-91da-79ac5828eef6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:31.193121 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7475d761-da1f-4b23-91da-79ac5828eef6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd84bdd46855d48fa9ff16513259028b2', 'request_id': 'req-7475d761-da1f-4b23-91da-79ac5828eef6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:31.199529 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7475d761-da1f-4b23-91da-79ac5828eef6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd84bdd46855d48fa9ff16513259028b2', 'request_id': 'req-7475d761-da1f-4b23-91da-79ac5828eef6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:31.205242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7475d761-da1f-4b23-91da-79ac5828eef6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd84bdd46855d48fa9ff16513259028b2', 'request_id': 'req-7475d761-da1f-4b23-91da-79ac5828eef6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:31.215481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:31.220163 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7475d761-da1f-4b23-91da-79ac5828eef6 admin admin] The token cache is being invalidate because project d84bdd46855d48fa9ff16513259028b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:31.220455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7475d761-da1f-4b23-91da-79ac5828eef6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:31.221587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7475d761-da1f-4b23-91da-79ac5828eef6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:31.223706 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1090/2163] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:31 2022] DELETE /identity/v3/projects/d84bdd46855d48fa9ff16513259028b2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:31.228342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c97b7a-c4ed-45c9-9952-2e0b0dd3c5e8 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:31.247123 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c97b7a-c4ed-45c9-9952-2e0b0dd3c5e8 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:31.303402 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4a6be-4fc0-4e4d-9d41-5fbeb3ef7e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:31.304264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4a6be-4fc0-4e4d-9d41-5fbeb3ef7e18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:31.304543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4a6be-4fc0-4e4d-9d41-5fbeb3ef7e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:31.304813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad4a6be-4fc0-4e4d-9d41-5fbeb3ef7e18 admin admin] PATH_INFO: `/v3/users/9dc75e7c4ccb44c29bb88f18bebebae7` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:31.306050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4a6be-4fc0-4e4d-9d41-5fbeb3ef7e18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dc75e7c4ccb44c29bb88f18bebebae7)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:31.307994 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad4a6be-4fc0-4e4d-9d41-5fbeb3ef7e18 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:31.351827 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-dad4a6be-4fc0-4e4d-9d41-5fbeb3ef7e18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dc75e7c4ccb44c29bb88f18bebebae7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:31.352479 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-dad4a6be-4fc0-4e4d-9d41-5fbeb3ef7e18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dc75e7c4ccb44c29bb88f18bebebae7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:31.364891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-dad4a6be-4fc0-4e4d-9d41-5fbeb3ef7e18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dc75e7c4ccb44c29bb88f18bebebae7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:31.372239 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-dad4a6be-4fc0-4e4d-9d41-5fbeb3ef7e18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dc75e7c4ccb44c29bb88f18bebebae7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:31.379236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-dad4a6be-4fc0-4e4d-9d41-5fbeb3ef7e18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dc75e7c4ccb44c29bb88f18bebebae7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:31.385153 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1074/2164] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:31 2022] DELETE /identity/v3/users/9dc75e7c4ccb44c29bb88f18bebebae7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:31.673422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475d761-da1f-4b23-91da-79ac5828eef6 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:31.674903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7475d761-da1f-4b23-91da-79ac5828eef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:31.691057 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475d761-da1f-4b23-91da-79ac5828eef6 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:31.756683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d86718-f38d-4bbd-af2b-7ee743e052b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:31.757784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d86718-f38d-4bbd-af2b-7ee743e052b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:31.758325 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d86718-f38d-4bbd-af2b-7ee743e052b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:31.758758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d86718-f38d-4bbd-af2b-7ee743e052b3 admin admin] PATH_INFO: `/v3/projects/da8039696c8849c2aef45dc909fdd475` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:31.760257 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d86718-f38d-4bbd-af2b-7ee743e052b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da8039696c8849c2aef45dc909fdd475)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:31.762744 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d86718-f38d-4bbd-af2b-7ee743e052b3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:31.836491 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-13d86718-f38d-4bbd-af2b-7ee743e052b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da8039696c8849c2aef45dc909fdd475', 'request_id': 'req-13d86718-f38d-4bbd-af2b-7ee743e052b3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:31.844792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-13d86718-f38d-4bbd-af2b-7ee743e052b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da8039696c8849c2aef45dc909fdd475', 'request_id': 'req-13d86718-f38d-4bbd-af2b-7ee743e052b3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:31.852132 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-13d86718-f38d-4bbd-af2b-7ee743e052b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da8039696c8849c2aef45dc909fdd475', 'request_id': 'req-13d86718-f38d-4bbd-af2b-7ee743e052b3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:31.868869 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-13d86718-f38d-4bbd-af2b-7ee743e052b3 admin admin] The token cache is being invalidate because project da8039696c8849c2aef45dc909fdd475 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:31.869194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-13d86718-f38d-4bbd-af2b-7ee743e052b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:31.869836 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-13d86718-f38d-4bbd-af2b-7ee743e052b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:31.871623 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1091/2165] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:31 2022] DELETE /identity/v3/projects/da8039696c8849c2aef45dc909fdd475 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:31.875585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4a6be-4fc0-4e4d-9d41-5fbeb3ef7e18 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:31.879073 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-dad4a6be-4fc0-4e4d-9d41-5fbeb3ef7e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:31.896467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad4a6be-4fc0-4e4d-9d41-5fbeb3ef7e18 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:31.986425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3231c-8393-4874-aacd-ddf6ffac4947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:31.989112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3231c-8393-4874-aacd-ddf6ffac4947 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:31.989112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3231c-8393-4874-aacd-ddf6ffac4947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:31.989112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3231c-8393-4874-aacd-ddf6ffac4947 admin admin] PATH_INFO: `/v3/users/ac3b65e5cbec437ba5e55a369eabc714` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:31.989600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3231c-8393-4874-aacd-ddf6ffac4947 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac3b65e5cbec437ba5e55a369eabc714)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:31.992223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3231c-8393-4874-aacd-ddf6ffac4947 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:32.001214 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:32.048358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2fa3231c-8393-4874-aacd-ddf6ffac4947 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac3b65e5cbec437ba5e55a369eabc714'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:32.049045 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2fa3231c-8393-4874-aacd-ddf6ffac4947 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac3b65e5cbec437ba5e55a369eabc714'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:32.061838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2fa3231c-8393-4874-aacd-ddf6ffac4947 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac3b65e5cbec437ba5e55a369eabc714'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:32.071421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2fa3231c-8393-4874-aacd-ddf6ffac4947 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac3b65e5cbec437ba5e55a369eabc714'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:32.079147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2fa3231c-8393-4874-aacd-ddf6ffac4947 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac3b65e5cbec437ba5e55a369eabc714'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:32.086136 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1075/2166] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:31 2022] DELETE /identity/v3/users/ac3b65e5cbec437ba5e55a369eabc714 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:32.724574 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d86718-f38d-4bbd-af2b-7ee743e052b3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:32.727242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-13d86718-f38d-4bbd-af2b-7ee743e052b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:32.740951 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d86718-f38d-4bbd-af2b-7ee743e052b3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:32.809125 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d87c7-092d-43c0-a6e6-5d85640a75e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:32.810023 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d87c7-092d-43c0-a6e6-5d85640a75e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:32.810306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d87c7-092d-43c0-a6e6-5d85640a75e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:32.810582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d87c7-092d-43c0-a6e6-5d85640a75e6 admin admin] PATH_INFO: `/v3/projects/23d6587610884bffaa0cb407a163df39` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:32.811775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d87c7-092d-43c0-a6e6-5d85640a75e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23d6587610884bffaa0cb407a163df39)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:32.813749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d87c7-092d-43c0-a6e6-5d85640a75e6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:32.851329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:32.867056 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b42d87c7-092d-43c0-a6e6-5d85640a75e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23d6587610884bffaa0cb407a163df39', 'request_id': 'req-b42d87c7-092d-43c0-a6e6-5d85640a75e6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:32.873702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b42d87c7-092d-43c0-a6e6-5d85640a75e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23d6587610884bffaa0cb407a163df39', 'request_id': 'req-b42d87c7-092d-43c0-a6e6-5d85640a75e6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:32.880508 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b42d87c7-092d-43c0-a6e6-5d85640a75e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23d6587610884bffaa0cb407a163df39', 'request_id': 'req-b42d87c7-092d-43c0-a6e6-5d85640a75e6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:32.893844 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b42d87c7-092d-43c0-a6e6-5d85640a75e6 admin admin] The token cache is being invalidate because project 23d6587610884bffaa0cb407a163df39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:32.894130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b42d87c7-092d-43c0-a6e6-5d85640a75e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:32.894762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b42d87c7-092d-43c0-a6e6-5d85640a75e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:32.897176 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1092/2167] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:32 2022] DELETE /identity/v3/projects/23d6587610884bffaa0cb407a163df39 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:32.903783 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efd31b-9709-47ab-a6b9-2543dddf179f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:32.903783 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efd31b-9709-47ab-a6b9-2543dddf179f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:32.903783 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9efd31b-9709-47ab-a6b9-2543dddf179f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:32.924081 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d9efd31b-9709-47ab-a6b9-2543dddf179f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:32.925632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d9efd31b-9709-47ab-a6b9-2543dddf179f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:32.994040 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1076/2168] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:33.002395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a918-3b05-4abd-a3db-42e58004de99 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:33.002994 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a918-3b05-4abd-a3db-42e58004de99 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:33.003556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893a918-3b05-4abd-a3db-42e58004de99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:33.025400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d893a918-3b05-4abd-a3db-42e58004de99 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:33.026734 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d893a918-3b05-4abd-a3db-42e58004de99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:33.035981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1093/2169] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:33.040067 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9efd31b-9709-47ab-a6b9-2543dddf179f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:33.050355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9efd31b-9709-47ab-a6b9-2543dddf179f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:33.056523 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e3aee-a7bb-4f42-9d71-9acda9190aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:33.057308 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e3aee-a7bb-4f42-9d71-9acda9190aac admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:33.057591 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e3aee-a7bb-4f42-9d71-9acda9190aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:33.057865 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e3aee-a7bb-4f42-9d71-9acda9190aac admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:33.058603 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e3aee-a7bb-4f42-9d71-9acda9190aac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:15:33.058914 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e3aee-a7bb-4f42-9d71-9acda9190aac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:33.060641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e3aee-a7bb-4f42-9d71-9acda9190aac admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:33.077558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1077/2170] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:15:33.081949 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893a918-3b05-4abd-a3db-42e58004de99 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:33.093960 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e3aee-a7bb-4f42-9d71-9acda9190aac admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:33.094548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893a918-3b05-4abd-a3db-42e58004de99 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:33.095896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d41e3aee-a7bb-4f42-9d71-9acda9190aac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:33.099343 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e845a-ceff-4438-a1fc-875de771e52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:33.100232 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e845a-ceff-4438-a1fc-875de771e52d admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:33.100509 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e845a-ceff-4438-a1fc-875de771e52d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:33.100814 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e845a-ceff-4438-a1fc-875de771e52d admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:33.101532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e845a-ceff-4438-a1fc-875de771e52d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:33.103421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e845a-ceff-4438-a1fc-875de771e52d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:33.108587 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e3aee-a7bb-4f42-9d71-9acda9190aac admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:33.114058 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617c986-9422-4806-9aea-4f95bd579f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:33.114931 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617c986-9422-4806-9aea-4f95bd579f84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:33.115758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617c986-9422-4806-9aea-4f95bd579f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:33.116065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617c986-9422-4806-9aea-4f95bd579f84 admin admin] PATH_INFO: `/v3/users/33d0db7375f94cedb7c8f8375c31d93f` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:33.117329 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617c986-9422-4806-9aea-4f95bd579f84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33d0db7375f94cedb7c8f8375c31d93f)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:33.119304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617c986-9422-4806-9aea-4f95bd579f84 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:33.128991 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1094/2171] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:33 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:33.133861 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e845a-ceff-4438-a1fc-875de771e52d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:33.167204 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9617c986-9422-4806-9aea-4f95bd579f84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33d0db7375f94cedb7c8f8375c31d93f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:33.167862 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9617c986-9422-4806-9aea-4f95bd579f84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33d0db7375f94cedb7c8f8375c31d93f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:33.180995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9617c986-9422-4806-9aea-4f95bd579f84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33d0db7375f94cedb7c8f8375c31d93f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:33.182957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:33.189999 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9617c986-9422-4806-9aea-4f95bd579f84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33d0db7375f94cedb7c8f8375c31d93f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:33.196839 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e845a-ceff-4438-a1fc-875de771e52d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:33.199492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9617c986-9422-4806-9aea-4f95bd579f84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33d0db7375f94cedb7c8f8375c31d93f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:33.201910 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83338dc9-b934-4148-b5b1-5b52bc6d9984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:33.202583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83338dc9-b934-4148-b5b1-5b52bc6d9984 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:33.202873 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83338dc9-b934-4148-b5b1-5b52bc6d9984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:33.203153 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83338dc9-b934-4148-b5b1-5b52bc6d9984 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:33.203871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83338dc9-b934-4148-b5b1-5b52bc6d9984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:33.205975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1078/2172] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:33 2022] DELETE /identity/v3/users/33d0db7375f94cedb7c8f8375c31d93f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:33.206086 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83338dc9-b934-4148-b5b1-5b52bc6d9984 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:33.210701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6a623-00de-48c2-8216-0db20ffc9339 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:33.210959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6a623-00de-48c2-8216-0db20ffc9339 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:33.211237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a6a623-00de-48c2-8216-0db20ffc9339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:33.228733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1095/2173] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:33 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:33.229311 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c3a6a623-00de-48c2-8216-0db20ffc9339 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:33.229892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83338dc9-b934-4148-b5b1-5b52bc6d9984 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:33.230524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c3a6a623-00de-48c2-8216-0db20ffc9339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:33.231490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-83338dc9-b934-4148-b5b1-5b52bc6d9984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:33.244044 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83338dc9-b934-4148-b5b1-5b52bc6d9984 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:33.298001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1079/2174] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:33.302349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792a7dc-1574-4666-a5d9-308150001e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:33.303592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792a7dc-1574-4666-a5d9-308150001e84 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:33.303987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792a7dc-1574-4666-a5d9-308150001e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:33.304266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792a7dc-1574-4666-a5d9-308150001e84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:33.305023 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792a7dc-1574-4666-a5d9-308150001e84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:33.306609 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0792a7dc-1574-4666-a5d9-308150001e84 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:33.308126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6a623-00de-48c2-8216-0db20ffc9339 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:33.309563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c3a6a623-00de-48c2-8216-0db20ffc9339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:33.316660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1096/2175] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:33.320531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792a7dc-1574-4666-a5d9-308150001e84 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:33.329787 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792a7dc-1574-4666-a5d9-308150001e84 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:33.334405 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef077a7-f108-46ca-aa37-485d57944209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:33.335041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef077a7-f108-46ca-aa37-485d57944209 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:33.335317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef077a7-f108-46ca-aa37-485d57944209 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:33.335586 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef077a7-f108-46ca-aa37-485d57944209 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:33.336299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef077a7-f108-46ca-aa37-485d57944209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:33.337887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef077a7-f108-46ca-aa37-485d57944209 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:33.347397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1097/2176] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:15:33.351982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef077a7-f108-46ca-aa37-485d57944209 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:33.361493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef077a7-f108-46ca-aa37-485d57944209 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:33.366930 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55e7b2-f8e5-4f49-b8b7-891620881e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:33.367089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a6a623-00de-48c2-8216-0db20ffc9339 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:33.367798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55e7b2-f8e5-4f49-b8b7-891620881e17 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:33.368083 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55e7b2-f8e5-4f49-b8b7-891620881e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:33.368358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d55e7b2-f8e5-4f49-b8b7-891620881e17 admin admin] PATH_INFO: `/v3/projects/7184107a4d6343d7ab15a46c128cc79b/users/3c2c0e2107744c10b0bfea8e19045d93/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:33.370628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:33.370741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8b85d-e6ba-46d6-80bc-e0cee9393d30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:33.371320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8b85d-e6ba-46d6-80bc-e0cee9393d30 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:33.371603 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8b85d-e6ba-46d6-80bc-e0cee9393d30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:33.371869 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8b85d-e6ba-46d6-80bc-e0cee9393d30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:33.382892 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8b85d-e6ba-46d6-80bc-e0cee9393d30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:33.385382 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8b85d-e6ba-46d6-80bc-e0cee9393d30 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:33.386652 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55e7b2-f8e5-4f49-b8b7-891620881e17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7184107a4d6343d7ab15a46c128cc79b, user_id=3c2c0e2107744c10b0bfea8e19045d93, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:33.389418 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d55e7b2-f8e5-4f49-b8b7-891620881e17 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:33.397342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1080/2177] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:15:33.410157 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1098/2178] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:33 2022] PUT /identity/v3/projects/7184107a4d6343d7ab15a46c128cc79b/users/3c2c0e2107744c10b0bfea8e19045d93/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:33.423037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c76675-0b16-419a-85f4-b73af9f7a062 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:33.423358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c76675-0b16-419a-85f4-b73af9f7a062 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:33.423639 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c76675-0b16-419a-85f4-b73af9f7a062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:33.455392 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-31c76675-0b16-419a-85f4-b73af9f7a062 None None] MFA Rules not processed for user `3c2c0e2107744c10b0bfea8e19045d93`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:33.456493 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-31c76675-0b16-419a-85f4-b73af9f7a062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:33.528886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1081/2179] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:33.533464 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55e7b2-f8e5-4f49-b8b7-891620881e17 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:33.613113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d55e7b2-f8e5-4f49-b8b7-891620881e17 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:33.618367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8e603-0986-49f1-9481-fcb84e07c56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:33.619027 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8e603-0986-49f1-9481-fcb84e07c56f admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:33.619306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8e603-0986-49f1-9481-fcb84e07c56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:33.619579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c8e603-0986-49f1-9481-fcb84e07c56f admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:33.620278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8e603-0986-49f1-9481-fcb84e07c56f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:33.622204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c8e603-0986-49f1-9481-fcb84e07c56f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:33.652620 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1099/2180] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:33 2022] POST /identity/v3/projects => generated 391 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:15:33.653818 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c76675-0b16-419a-85f4-b73af9f7a062 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:33.666145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c76675-0b16-419a-85f4-b73af9f7a062 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:33.680772 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:33.693225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8e603-0986-49f1-9481-fcb84e07c56f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:33.695601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-22c8e603-0986-49f1-9481-fcb84e07c56f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:33.776524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d9f34-3b87-42f5-bffc-519ff2cefeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:33.777631 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d9f34-3b87-42f5-bffc-519ff2cefeb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:33.778025 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d9f34-3b87-42f5-bffc-519ff2cefeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:33.778485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d9f34-3b87-42f5-bffc-519ff2cefeb8 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:33.779282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d9f34-3b87-42f5-bffc-519ff2cefeb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:33.781645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d9f34-3b87-42f5-bffc-519ff2cefeb8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:33.784542 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c8e603-0986-49f1-9481-fcb84e07c56f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:33.792957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:33.794536 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:33.794987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:33.795456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78 admin admin] PATH_INFO: `/v3/projects/45a2bc23fc1f45f29d2e497fe5dbd482` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:33.796756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45a2bc23fc1f45f29d2e497fe5dbd482)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:33.798694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:33.815133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1082/2181] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:33 2022] POST /identity/v3/users => generated 302 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:33.820445 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d9f34-3b87-42f5-bffc-519ff2cefeb8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:33.834807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d9f34-3b87-42f5-bffc-519ff2cefeb8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:33.843013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49095246-4f83-4540-b60a-92d3b8fdde83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:33.843760 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49095246-4f83-4540-b60a-92d3b8fdde83 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:33.844045 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49095246-4f83-4540-b60a-92d3b8fdde83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:33.845028 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49095246-4f83-4540-b60a-92d3b8fdde83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:33.845868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49095246-4f83-4540-b60a-92d3b8fdde83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:33.848147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49095246-4f83-4540-b60a-92d3b8fdde83 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:33.861327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45a2bc23fc1f45f29d2e497fe5dbd482', 'request_id': 'req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:33.869353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45a2bc23fc1f45f29d2e497fe5dbd482', 'request_id': 'req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:33.874158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1083/2182] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:33.877197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45a2bc23fc1f45f29d2e497fe5dbd482', 'request_id': 'req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:33.881698 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49095246-4f83-4540-b60a-92d3b8fdde83 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:33.890751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:33.895549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49095246-4f83-4540-b60a-92d3b8fdde83 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:33.896198 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78 admin admin] The token cache is being invalidate because project 45a2bc23fc1f45f29d2e497fe5dbd482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:33.897669 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:33.898379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:33.903293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1100/2183] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:33 2022] DELETE /identity/v3/projects/45a2bc23fc1f45f29d2e497fe5dbd482 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:33.906158 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:33.908699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:34.002312 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0ab2d-0f6a-42c0-888e-4900be795e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:34.003385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0ab2d-0f6a-42c0-888e-4900be795e0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:34.003717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec078c1-4e6a-4ef6-8b99-c2e713f46e78 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:34.003835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0ab2d-0f6a-42c0-888e-4900be795e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:34.004216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0ab2d-0f6a-42c0-888e-4900be795e0a admin admin] PATH_INFO: `/v3/projects/7c1629adbcbe4996af371783f159be17/users/b3f4a80731c044399070459b99df2901/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:34.009970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efd52a-5b15-451d-bda1-f69c3630bddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:34.011110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efd52a-5b15-451d-bda1-f69c3630bddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:34.011408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efd52a-5b15-451d-bda1-f69c3630bddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:34.012082 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3efd52a-5b15-451d-bda1-f69c3630bddb admin admin] PATH_INFO: `/v3/users/4987b5aa0eb44ac5b70633f5fc7b35d1` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:34.013674 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efd52a-5b15-451d-bda1-f69c3630bddb admin admin] RBAC: Authorizing `identity:delete_user(user_id=4987b5aa0eb44ac5b70633f5fc7b35d1)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:34.015472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3efd52a-5b15-451d-bda1-f69c3630bddb admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:34.024111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0ab2d-0f6a-42c0-888e-4900be795e0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c1629adbcbe4996af371783f159be17, user_id=b3f4a80731c044399070459b99df2901, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:34.026959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0ab2d-0f6a-42c0-888e-4900be795e0a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:34.044701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1084/2184] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:33 2022] PUT /identity/v3/projects/7c1629adbcbe4996af371783f159be17/users/b3f4a80731c044399070459b99df2901/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:34.049190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0ab2d-0f6a-42c0-888e-4900be795e0a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:34.050869 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b2a0ab2d-0f6a-42c0-888e-4900be795e0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:34.078566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c3efd52a-5b15-451d-bda1-f69c3630bddb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4987b5aa0eb44ac5b70633f5fc7b35d1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:34.079223 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c3efd52a-5b15-451d-bda1-f69c3630bddb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4987b5aa0eb44ac5b70633f5fc7b35d1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:34.092396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c3efd52a-5b15-451d-bda1-f69c3630bddb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4987b5aa0eb44ac5b70633f5fc7b35d1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:34.104530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c3efd52a-5b15-451d-bda1-f69c3630bddb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4987b5aa0eb44ac5b70633f5fc7b35d1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:34.113868 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c3efd52a-5b15-451d-bda1-f69c3630bddb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4987b5aa0eb44ac5b70633f5fc7b35d1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:34.120367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1101/2185] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:33 2022] DELETE /identity/v3/users/4987b5aa0eb44ac5b70633f5fc7b35d1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:34.128347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0ab2d-0f6a-42c0-888e-4900be795e0a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:34.153074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:34.187591 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e1f6c2-a427-4c5c-b9c2-c4bf81d770b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:34.188252 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1f6c2-a427-4c5c-b9c2-c4bf81d770b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:34.188527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1f6c2-a427-4c5c-b9c2-c4bf81d770b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:34.188796 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e1f6c2-a427-4c5c-b9c2-c4bf81d770b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:34.189556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1f6c2-a427-4c5c-b9c2-c4bf81d770b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:34.191185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e1f6c2-a427-4c5c-b9c2-c4bf81d770b2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:34.201803 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1085/2186] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:34.206111 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efd52a-5b15-451d-bda1-f69c3630bddb admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:34.207495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c3efd52a-5b15-451d-bda1-f69c3630bddb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:34.221787 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3efd52a-5b15-451d-bda1-f69c3630bddb admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:34.227110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ce44a-999d-4668-9f7f-46ef2bd6e06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:34.228000 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ce44a-999d-4668-9f7f-46ef2bd6e06d admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:34.228281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ce44a-999d-4668-9f7f-46ef2bd6e06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:34.228555 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23ce44a-999d-4668-9f7f-46ef2bd6e06d admin admin] PATH_INFO: `/v3/projects/7c1629adbcbe4996af371783f159be17/users/b3f4a80731c044399070459b99df2901/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:34.230697 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ce44a-999d-4668-9f7f-46ef2bd6e06d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c1629adbcbe4996af371783f159be17, user_id=b3f4a80731c044399070459b99df2901, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:34.233389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23ce44a-999d-4668-9f7f-46ef2bd6e06d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:34.249761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1102/2187] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:34 2022] PUT /identity/v3/projects/7c1629adbcbe4996af371783f159be17/users/b3f4a80731c044399070459b99df2901/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:34.254651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6942d96-0c32-471b-849e-52b43b81f147 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:34.254942 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6942d96-0c32-471b-849e-52b43b81f147 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:34.255221 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6942d96-0c32-471b-849e-52b43b81f147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:34.284534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b6942d96-0c32-471b-849e-52b43b81f147 None None] MFA Rules not processed for user `b3f4a80731c044399070459b99df2901`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:34.285885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b6942d96-0c32-471b-849e-52b43b81f147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:34.356422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1086/2188] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:34.367793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ce44a-999d-4668-9f7f-46ef2bd6e06d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:34.369506 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b23ce44a-999d-4668-9f7f-46ef2bd6e06d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:34.395986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6942d96-0c32-471b-849e-52b43b81f147 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:34.398548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b6942d96-0c32-471b-849e-52b43b81f147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:34.440287 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23ce44a-999d-4668-9f7f-46ef2bd6e06d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:34.444542 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe99651-1e48-4b0c-bb4f-12d9db47b125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:34.445214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe99651-1e48-4b0c-bb4f-12d9db47b125 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:34.445495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe99651-1e48-4b0c-bb4f-12d9db47b125 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:34.445767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe99651-1e48-4b0c-bb4f-12d9db47b125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:34.455574 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe99651-1e48-4b0c-bb4f-12d9db47b125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:34.457088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe99651-1e48-4b0c-bb4f-12d9db47b125 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:34.466897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1103/2189] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:34.480731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6942d96-0c32-471b-849e-52b43b81f147 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:34.485593 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:34.486452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:34.486730 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:34.487006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3 admin admin] PATH_INFO: `/v3/projects/3b3dd50d1e57413b829e8d62ce1049c7` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:34.488267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b3dd50d1e57413b829e8d62ce1049c7)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:34.490738 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:34.514762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe99651-1e48-4b0c-bb4f-12d9db47b125 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:34.516288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ffe99651-1e48-4b0c-bb4f-12d9db47b125 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:34.529720 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe99651-1e48-4b0c-bb4f-12d9db47b125 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:34.533604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:34.543642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b3dd50d1e57413b829e8d62ce1049c7', 'request_id': 'req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:34.544918 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:34.555415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b3dd50d1e57413b829e8d62ce1049c7', 'request_id': 'req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:34.565983 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b3dd50d1e57413b829e8d62ce1049c7', 'request_id': 'req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:34.577581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3 admin admin] The token cache is being invalidate because project 3b3dd50d1e57413b829e8d62ce1049c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:34.577867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:34.578518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:34.580198 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1087/2190] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:34 2022] DELETE /identity/v3/projects/3b3dd50d1e57413b829e8d62ce1049c7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:34.582412 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7758b2-e9cf-4bc9-9bac-e7f21cd30768 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:34.583065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7758b2-e9cf-4bc9-9bac-e7f21cd30768 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:34.583348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7758b2-e9cf-4bc9-9bac-e7f21cd30768 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:34.583625 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7758b2-e9cf-4bc9-9bac-e7f21cd30768 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:34.583893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:34.586116 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:34.586670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7b7758b2-e9cf-4bc9-9bac-e7f21cd30768 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:34.647779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7758b2-e9cf-4bc9-9bac-e7f21cd30768 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:34.649258 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7758b2-e9cf-4bc9-9bac-e7f21cd30768 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:34.652348 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eaf8c9-6895-4ad5-b24a-b52b59f362c3 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:34.657068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995a5157-715b-4aed-bcf8-c84cfb7278d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:34.657941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a5157-715b-4aed-bcf8-c84cfb7278d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:34.658219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a5157-715b-4aed-bcf8-c84cfb7278d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:34.658492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995a5157-715b-4aed-bcf8-c84cfb7278d5 admin admin] PATH_INFO: `/v3/users/f9fe5e98b83244379452ad5d312c06ad` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:34.658773 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1104/2191] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:15:34.659715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a5157-715b-4aed-bcf8-c84cfb7278d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9fe5e98b83244379452ad5d312c06ad)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:34.661748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995a5157-715b-4aed-bcf8-c84cfb7278d5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:34.713940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-995a5157-715b-4aed-bcf8-c84cfb7278d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9fe5e98b83244379452ad5d312c06ad'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:34.714562 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-995a5157-715b-4aed-bcf8-c84cfb7278d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9fe5e98b83244379452ad5d312c06ad'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:34.726579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-995a5157-715b-4aed-bcf8-c84cfb7278d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9fe5e98b83244379452ad5d312c06ad'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:34.733711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-995a5157-715b-4aed-bcf8-c84cfb7278d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9fe5e98b83244379452ad5d312c06ad'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:34.742861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-995a5157-715b-4aed-bcf8-c84cfb7278d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9fe5e98b83244379452ad5d312c06ad'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:34.749173 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1088/2192] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:34 2022] DELETE /identity/v3/users/f9fe5e98b83244379452ad5d312c06ad => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:35.159688 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7758b2-e9cf-4bc9-9bac-e7f21cd30768 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:35.171250 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7758b2-e9cf-4bc9-9bac-e7f21cd30768 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:35.247471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00921c63-4921-4268-9368-23125fc1dfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:35.248474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00921c63-4921-4268-9368-23125fc1dfc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:35.248823 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00921c63-4921-4268-9368-23125fc1dfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:35.249193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00921c63-4921-4268-9368-23125fc1dfc4 admin admin] PATH_INFO: `/v3/projects/f68a3a7c657646f6b46d382f052469b9` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:35.256987 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00921c63-4921-4268-9368-23125fc1dfc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f68a3a7c657646f6b46d382f052469b9)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:35.259053 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00921c63-4921-4268-9368-23125fc1dfc4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:35.297057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2c055-05aa-43c8-97ff-5379c0810e57 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:35.297791 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2c055-05aa-43c8-97ff-5379c0810e57 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:35.297791 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d2c055-05aa-43c8-97ff-5379c0810e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:35.316215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-39d2c055-05aa-43c8-97ff-5379c0810e57 None None] MFA Rules not processed for user `5298be027cf843638957c4a15003db7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:35.317893 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-00921c63-4921-4268-9368-23125fc1dfc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f68a3a7c657646f6b46d382f052469b9', 'request_id': 'req-00921c63-4921-4268-9368-23125fc1dfc4'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:35.318263 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-39d2c055-05aa-43c8-97ff-5379c0810e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:35.324124 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-00921c63-4921-4268-9368-23125fc1dfc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f68a3a7c657646f6b46d382f052469b9', 'request_id': 'req-00921c63-4921-4268-9368-23125fc1dfc4'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:35.330148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-00921c63-4921-4268-9368-23125fc1dfc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f68a3a7c657646f6b46d382f052469b9', 'request_id': 'req-00921c63-4921-4268-9368-23125fc1dfc4'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:35.331809 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:35.345262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-00921c63-4921-4268-9368-23125fc1dfc4 admin admin] The token cache is being invalidate because project f68a3a7c657646f6b46d382f052469b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:35.345624 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-00921c63-4921-4268-9368-23125fc1dfc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:35.346695 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-00921c63-4921-4268-9368-23125fc1dfc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:35.348306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1105/2193] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:35 2022] DELETE /identity/v3/projects/f68a3a7c657646f6b46d382f052469b9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:35.354801 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb472bdd-fc07-4ebe-a16e-4d3afb5d0a62 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:35.355155 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb472bdd-fc07-4ebe-a16e-4d3afb5d0a62 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:35.355501 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb472bdd-fc07-4ebe-a16e-4d3afb5d0a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:35.373239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-fb472bdd-fc07-4ebe-a16e-4d3afb5d0a62 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:35.374417 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fb472bdd-fc07-4ebe-a16e-4d3afb5d0a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:35.376964 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1089/2194] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:15:35.436965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1106/2195] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:35.442925 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0807c3-8069-4f97-b9ea-2f1a1962c67e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:35.443215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0807c3-8069-4f97-b9ea-2f1a1962c67e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:35.443531 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0807c3-8069-4f97-b9ea-2f1a1962c67e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:35.450050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:35.468447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9e0807c3-8069-4f97-b9ea-2f1a1962c67e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:35.470063 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9e0807c3-8069-4f97-b9ea-2f1a1962c67e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:35.479487 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1090/2196] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:35.484226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb472bdd-fc07-4ebe-a16e-4d3afb5d0a62 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:35.493963 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb472bdd-fc07-4ebe-a16e-4d3afb5d0a62 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:35.498846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65c0ed-acb7-4053-a8ce-39b0ed89da97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:35.499498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65c0ed-acb7-4053-a8ce-39b0ed89da97 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:35.499775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65c0ed-acb7-4053-a8ce-39b0ed89da97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:35.500044 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65c0ed-acb7-4053-a8ce-39b0ed89da97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:35.500764 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65c0ed-acb7-4053-a8ce-39b0ed89da97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:15:35.501106 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65c0ed-acb7-4053-a8ce-39b0ed89da97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:35.502682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d65c0ed-acb7-4053-a8ce-39b0ed89da97 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:35.518768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1107/2197] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:15:35.520195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0807c3-8069-4f97-b9ea-2f1a1962c67e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:35.530465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0807c3-8069-4f97-b9ea-2f1a1962c67e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:35.535187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262d603-0f9e-4c60-8991-9c05b34c187e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:35.535839 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262d603-0f9e-4c60-8991-9c05b34c187e admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:35.536123 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262d603-0f9e-4c60-8991-9c05b34c187e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:35.536396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262d603-0f9e-4c60-8991-9c05b34c187e admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:35.537187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262d603-0f9e-4c60-8991-9c05b34c187e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:35.539105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b262d603-0f9e-4c60-8991-9c05b34c187e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:35.565476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1091/2198] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:35 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:35.570439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65c0ed-acb7-4053-a8ce-39b0ed89da97 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:35.580802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65c0ed-acb7-4053-a8ce-39b0ed89da97 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:35.639338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2966ef4-bb4a-4f1d-a86c-56d571032277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:35.640018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2966ef4-bb4a-4f1d-a86c-56d571032277 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:35.640298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2966ef4-bb4a-4f1d-a86c-56d571032277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:35.640575 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2966ef4-bb4a-4f1d-a86c-56d571032277 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:35.641322 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2966ef4-bb4a-4f1d-a86c-56d571032277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:35.643246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2966ef4-bb4a-4f1d-a86c-56d571032277 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:35.667966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1108/2199] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:35 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:15:35.669362 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262d603-0f9e-4c60-8991-9c05b34c187e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:35.681396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262d603-0f9e-4c60-8991-9c05b34c187e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:35.686217 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc758b43-ff55-4857-bae3-f0bfd85d4af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:35.686895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc758b43-ff55-4857-bae3-f0bfd85d4af6 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:35.687171 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc758b43-ff55-4857-bae3-f0bfd85d4af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:35.687441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc758b43-ff55-4857-bae3-f0bfd85d4af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:35.688183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc758b43-ff55-4857-bae3-f0bfd85d4af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:35.689818 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc758b43-ff55-4857-bae3-f0bfd85d4af6 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:35.702140 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1092/2200] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:35.706311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2966ef4-bb4a-4f1d-a86c-56d571032277 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:35.717502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2966ef4-bb4a-4f1d-a86c-56d571032277 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:35.722246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28ca05-dcf5-4024-b788-4f7233f37b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:35.722934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28ca05-dcf5-4024-b788-4f7233f37b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:35.723218 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28ca05-dcf5-4024-b788-4f7233f37b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:35.723493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28ca05-dcf5-4024-b788-4f7233f37b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:35.724241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28ca05-dcf5-4024-b788-4f7233f37b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:35.726446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28ca05-dcf5-4024-b788-4f7233f37b58 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:35.741370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1109/2201] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:35.742785 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc758b43-ff55-4857-bae3-f0bfd85d4af6 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:35.753021 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc758b43-ff55-4857-bae3-f0bfd85d4af6 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:35.758423 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865c660-81d6-4e57-b701-2f3828216b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:35.759332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865c660-81d6-4e57-b701-2f3828216b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:35.759611 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865c660-81d6-4e57-b701-2f3828216b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:35.759885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a865c660-81d6-4e57-b701-2f3828216b06 admin admin] PATH_INFO: `/v3/projects/4c6e169f09a3410084ca68a4a9fc9956/users/938620e3c8064c59b5c316bfcf535cbd/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:35.777780 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a865c660-81d6-4e57-b701-2f3828216b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6e169f09a3410084ca68a4a9fc9956, user_id=938620e3c8064c59b5c316bfcf535cbd, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:35.780514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a865c660-81d6-4e57-b701-2f3828216b06 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:35.794337 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1093/2202] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:35 2022] PUT /identity/v3/projects/4c6e169f09a3410084ca68a4a9fc9956/users/938620e3c8064c59b5c316bfcf535cbd/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:35.806699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5fbbe-9bbf-4f10-a100-970ff8b7877f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:35.807084 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5fbbe-9bbf-4f10-a100-970ff8b7877f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:35.807418 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5fbbe-9bbf-4f10-a100-970ff8b7877f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:35.836497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-40c5fbbe-9bbf-4f10-a100-970ff8b7877f None None] MFA Rules not processed for user `938620e3c8064c59b5c316bfcf535cbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:35.837747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-40c5fbbe-9bbf-4f10-a100-970ff8b7877f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:35.891301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:35.918892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1110/2203] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:35.933934 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865c660-81d6-4e57-b701-2f3828216b06 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:35.935821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a865c660-81d6-4e57-b701-2f3828216b06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:36.028966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a865c660-81d6-4e57-b701-2f3828216b06 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:36.033776 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25a7845-4af1-40fe-93fc-5fb388ab05c2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b5660cfd344dcd8abe14c88df686e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:36.034065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a7845-4af1-40fe-93fc-5fb388ab05c2 service glance] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:36.034483 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a7845-4af1-40fe-93fc-5fb388ab05c2 service glance] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:36.034863 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25a7845-4af1-40fe-93fc-5fb388ab05c2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:36.046020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a7845-4af1-40fe-93fc-5fb388ab05c2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:36.047605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25a7845-4af1-40fe-93fc-5fb388ab05c2 service glance] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:36.058716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1094/2204] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:35 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:38.314460 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5fbbe-9bbf-4f10-a100-970ff8b7877f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:38.315919 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-40c5fbbe-9bbf-4f10-a100-970ff8b7877f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:38.387178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5fbbe-9bbf-4f10-a100-970ff8b7877f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:38.391708 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8f6d1-6e7c-4847-98e5-df03bf9d3036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:38.392585 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8f6d1-6e7c-4847-98e5-df03bf9d3036 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:38.392892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8f6d1-6e7c-4847-98e5-df03bf9d3036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:38.393197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba8f6d1-6e7c-4847-98e5-df03bf9d3036 admin admin] PATH_INFO: `/v3/users/3c2c0e2107744c10b0bfea8e19045d93` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:38.394537 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba8f6d1-6e7c-4847-98e5-df03bf9d3036 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c2c0e2107744c10b0bfea8e19045d93)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:38.397071 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba8f6d1-6e7c-4847-98e5-df03bf9d3036 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:38.461267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9ba8f6d1-6e7c-4847-98e5-df03bf9d3036 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c2c0e2107744c10b0bfea8e19045d93'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:38.462902 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9ba8f6d1-6e7c-4847-98e5-df03bf9d3036 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c2c0e2107744c10b0bfea8e19045d93'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:38.486228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9ba8f6d1-6e7c-4847-98e5-df03bf9d3036 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c2c0e2107744c10b0bfea8e19045d93'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:38.497040 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9ba8f6d1-6e7c-4847-98e5-df03bf9d3036 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c2c0e2107744c10b0bfea8e19045d93'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:38.507993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9ba8f6d1-6e7c-4847-98e5-df03bf9d3036 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c2c0e2107744c10b0bfea8e19045d93'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:38.518444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1111/2205] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:38 2022] DELETE /identity/v3/users/3c2c0e2107744c10b0bfea8e19045d93 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:38.524220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597eca0-df3a-4a98-ac01-731a56b126b7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:38.524683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597eca0-df3a-4a98-ac01-731a56b126b7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:38.524794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597eca0-df3a-4a98-ac01-731a56b126b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:38.542353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4597eca0-df3a-4a98-ac01-731a56b126b7 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:38.543557 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4597eca0-df3a-4a98-ac01-731a56b126b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:38.604337 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1095/2206] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:38.613916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8f6d1-6e7c-4847-98e5-df03bf9d3036 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:38.615354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9ba8f6d1-6e7c-4847-98e5-df03bf9d3036 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:38.683068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba8f6d1-6e7c-4847-98e5-df03bf9d3036 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:38.686693 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136b766-2d14-4c1b-9a3a-5f0f9e1ecce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:38.687336 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136b766-2d14-4c1b-9a3a-5f0f9e1ecce1 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:38.687612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136b766-2d14-4c1b-9a3a-5f0f9e1ecce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:38.687880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0136b766-2d14-4c1b-9a3a-5f0f9e1ecce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:38.697175 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136b766-2d14-4c1b-9a3a-5f0f9e1ecce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:38.698607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0136b766-2d14-4c1b-9a3a-5f0f9e1ecce1 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:38.707915 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1112/2207] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:39.025444 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597eca0-df3a-4a98-ac01-731a56b126b7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:39.028215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4597eca0-df3a-4a98-ac01-731a56b126b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:39.043747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597eca0-df3a-4a98-ac01-731a56b126b7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:39.048425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:39.055761 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:39.056082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:39.056369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef admin admin] PATH_INFO: `/v3/projects/7184107a4d6343d7ab15a46c128cc79b` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:39.057771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=7184107a4d6343d7ab15a46c128cc79b)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:39.059666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:39.081577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:39.114532 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7184107a4d6343d7ab15a46c128cc79b', 'request_id': 'req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:39.121264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7184107a4d6343d7ab15a46c128cc79b', 'request_id': 'req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:39.129726 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7184107a4d6343d7ab15a46c128cc79b', 'request_id': 'req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:39.142691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef admin admin] The token cache is being invalidate because project 7184107a4d6343d7ab15a46c128cc79b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:39.142691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:39.143278 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:39.144820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1096/2208] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:39 2022] DELETE /identity/v3/projects/7184107a4d6343d7ab15a46c128cc79b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:39.148948 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136b766-2d14-4c1b-9a3a-5f0f9e1ecce1 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:39.160704 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0136b766-2d14-4c1b-9a3a-5f0f9e1ecce1 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:39.166581 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:39.250539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760c3df-b4e7-476e-9e74-b4a0871819ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:39.251457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760c3df-b4e7-476e-9e74-b4a0871819ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:39.251741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760c3df-b4e7-476e-9e74-b4a0871819ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:39.252153 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760c3df-b4e7-476e-9e74-b4a0871819ad admin admin] PATH_INFO: `/v3/users/b3f4a80731c044399070459b99df2901` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:39.253404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760c3df-b4e7-476e-9e74-b4a0871819ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f4a80731c044399070459b99df2901)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:39.255339 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760c3df-b4e7-476e-9e74-b4a0871819ad admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:39.300254 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9760c3df-b4e7-476e-9e74-b4a0871819ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f4a80731c044399070459b99df2901'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:39.300899 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9760c3df-b4e7-476e-9e74-b4a0871819ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f4a80731c044399070459b99df2901'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:39.313434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9760c3df-b4e7-476e-9e74-b4a0871819ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f4a80731c044399070459b99df2901'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:39.320294 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9760c3df-b4e7-476e-9e74-b4a0871819ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f4a80731c044399070459b99df2901'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:39.327259 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9760c3df-b4e7-476e-9e74-b4a0871819ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f4a80731c044399070459b99df2901'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:39.333143 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1113/2209] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:39 2022] DELETE /identity/v3/users/b3f4a80731c044399070459b99df2901 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:39.642109 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:39.643551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:39.658147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d01f24-23eb-4119-bee3-9f4b24ce91ef admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:39.725477 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b08356f-31a9-4aa9-b384-a7e487889c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:39.726407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08356f-31a9-4aa9-b384-a7e487889c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:39.726682 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08356f-31a9-4aa9-b384-a7e487889c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:39.726952 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b08356f-31a9-4aa9-b384-a7e487889c7a admin admin] PATH_INFO: `/v3/projects/7c1629adbcbe4996af371783f159be17` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:39.728190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08356f-31a9-4aa9-b384-a7e487889c7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c1629adbcbe4996af371783f159be17)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:39.730166 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b08356f-31a9-4aa9-b384-a7e487889c7a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:39.776762 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4b08356f-31a9-4aa9-b384-a7e487889c7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c1629adbcbe4996af371783f159be17', 'request_id': 'req-4b08356f-31a9-4aa9-b384-a7e487889c7a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:39.783234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4b08356f-31a9-4aa9-b384-a7e487889c7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c1629adbcbe4996af371783f159be17', 'request_id': 'req-4b08356f-31a9-4aa9-b384-a7e487889c7a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:39.791279 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4b08356f-31a9-4aa9-b384-a7e487889c7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c1629adbcbe4996af371783f159be17', 'request_id': 'req-4b08356f-31a9-4aa9-b384-a7e487889c7a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:39.801812 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4b08356f-31a9-4aa9-b384-a7e487889c7a admin admin] The token cache is being invalidate because project 7c1629adbcbe4996af371783f159be17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:39.802097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4b08356f-31a9-4aa9-b384-a7e487889c7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:39.802889 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4b08356f-31a9-4aa9-b384-a7e487889c7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:39.804447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1097/2210] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:39 2022] DELETE /identity/v3/projects/7c1629adbcbe4996af371783f159be17 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:39.809299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f04cb-1201-4f4b-8154-5f7a5cae1015 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:39.809592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f04cb-1201-4f4b-8154-5f7a5cae1015 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:39.809872 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f04cb-1201-4f4b-8154-5f7a5cae1015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:39.828220 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-525f04cb-1201-4f4b-8154-5f7a5cae1015 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:39.829479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-525f04cb-1201-4f4b-8154-5f7a5cae1015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:39.897441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1114/2211] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:39.905610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7462b02-9324-47a6-911a-df01b4cafa40 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:39.905978 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7462b02-9324-47a6-911a-df01b4cafa40 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:39.906344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7462b02-9324-47a6-911a-df01b4cafa40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:39.931863 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f7462b02-9324-47a6-911a-df01b4cafa40 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:39.933234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f7462b02-9324-47a6-911a-df01b4cafa40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:39.944979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1098/2212] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:39.948152 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f04cb-1201-4f4b-8154-5f7a5cae1015 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:39.975354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f04cb-1201-4f4b-8154-5f7a5cae1015 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:39.980402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f1eba-9d12-4b84-a0cf-2c988be21821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:39.981224 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f1eba-9d12-4b84-a0cf-2c988be21821 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:39.981441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f1eba-9d12-4b84-a0cf-2c988be21821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:39.981718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700f1eba-9d12-4b84-a0cf-2c988be21821 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:39.982601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f1eba-9d12-4b84-a0cf-2c988be21821 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:15:39.982979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f1eba-9d12-4b84-a0cf-2c988be21821 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:39.984582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700f1eba-9d12-4b84-a0cf-2c988be21821 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:40.005173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1115/2213] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:15:40.007100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7462b02-9324-47a6-911a-df01b4cafa40 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:40.016265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7462b02-9324-47a6-911a-df01b4cafa40 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:40.021041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5f1d0-3ed8-4875-b38e-1d505e824732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:40.022304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5f1d0-3ed8-4875-b38e-1d505e824732 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:40.022576 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5f1d0-3ed8-4875-b38e-1d505e824732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:40.022846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5f1d0-3ed8-4875-b38e-1d505e824732 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:40.023560 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5f1d0-3ed8-4875-b38e-1d505e824732 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:40.025469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5f1d0-3ed8-4875-b38e-1d505e824732 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:40.052048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1099/2214] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:40 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:40.056826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f1eba-9d12-4b84-a0cf-2c988be21821 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:40.067697 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700f1eba-9d12-4b84-a0cf-2c988be21821 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:40.080474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:40.128111 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97598c2-fcd7-4c3c-bf74-cfebee57cc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:40.128782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97598c2-fcd7-4c3c-bf74-cfebee57cc5f admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:40.129109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97598c2-fcd7-4c3c-bf74-cfebee57cc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:40.129388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97598c2-fcd7-4c3c-bf74-cfebee57cc5f admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:40.130114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97598c2-fcd7-4c3c-bf74-cfebee57cc5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:40.132047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97598c2-fcd7-4c3c-bf74-cfebee57cc5f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:40.155273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1116/2215] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:40 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:40.156645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5f1d0-3ed8-4875-b38e-1d505e824732 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:40.168545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5f1d0-3ed8-4875-b38e-1d505e824732 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:40.173566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a18f2-80a6-44e7-abb8-602cfe6f3c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:40.174241 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a18f2-80a6-44e7-abb8-602cfe6f3c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:40.174527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a18f2-80a6-44e7-abb8-602cfe6f3c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:40.174804 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a18f2-80a6-44e7-abb8-602cfe6f3c1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:40.175567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a18f2-80a6-44e7-abb8-602cfe6f3c1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:40.177214 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8a18f2-80a6-44e7-abb8-602cfe6f3c1d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:40.179338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:40.193985 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1100/2216] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:40.199464 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97598c2-fcd7-4c3c-bf74-cfebee57cc5f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:40.209582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97598c2-fcd7-4c3c-bf74-cfebee57cc5f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:40.214288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3f250-25cf-44fe-8ceb-e2301b17ab4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:40.214975 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3f250-25cf-44fe-8ceb-e2301b17ab4e admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:40.215256 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3f250-25cf-44fe-8ceb-e2301b17ab4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:40.215531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3f250-25cf-44fe-8ceb-e2301b17ab4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:40.216289 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3f250-25cf-44fe-8ceb-e2301b17ab4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:40.217909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3f250-25cf-44fe-8ceb-e2301b17ab4e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:40.229348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1117/2217] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:40.233317 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a18f2-80a6-44e7-abb8-602cfe6f3c1d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:40.251610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a18f2-80a6-44e7-abb8-602cfe6f3c1d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:40.256541 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293e417-621f-4f97-86c4-148c5e1d8e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:40.257602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293e417-621f-4f97-86c4-148c5e1d8e15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:40.258065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293e417-621f-4f97-86c4-148c5e1d8e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:40.258356 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293e417-621f-4f97-86c4-148c5e1d8e15 admin admin] PATH_INFO: `/v3/projects/7f452e62b7494a3da6953db48cc14969/users/ad16f4273b0b42ad8c362226cbde84c5/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:40.275535 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293e417-621f-4f97-86c4-148c5e1d8e15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f452e62b7494a3da6953db48cc14969, user_id=ad16f4273b0b42ad8c362226cbde84c5, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:40.278549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293e417-621f-4f97-86c4-148c5e1d8e15 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:40.293175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1101/2218] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:40 2022] PUT /identity/v3/projects/7f452e62b7494a3da6953db48cc14969/users/ad16f4273b0b42ad8c362226cbde84c5/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:40.304848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535c6bb-0d81-4a65-bffe-77990ab43428 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:40.305240 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535c6bb-0d81-4a65-bffe-77990ab43428 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:40.305586 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535c6bb-0d81-4a65-bffe-77990ab43428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:40.332349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6535c6bb-0d81-4a65-bffe-77990ab43428 None None] MFA Rules not processed for user `ad16f4273b0b42ad8c362226cbde84c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:40.333642 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6535c6bb-0d81-4a65-bffe-77990ab43428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:40.406330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1118/2219] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:15:40.410684 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293e417-621f-4f97-86c4-148c5e1d8e15 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:40.483763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293e417-621f-4f97-86c4-148c5e1d8e15 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:40.488821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905df1cc-40e6-45a0-9686-dced2b59ae66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:40.489513 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905df1cc-40e6-45a0-9686-dced2b59ae66 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:40.489820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905df1cc-40e6-45a0-9686-dced2b59ae66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:40.490129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905df1cc-40e6-45a0-9686-dced2b59ae66 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:40.490858 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905df1cc-40e6-45a0-9686-dced2b59ae66 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:40.492763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905df1cc-40e6-45a0-9686-dced2b59ae66 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:40.517849 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1102/2220] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:40 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:40.522372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535c6bb-0d81-4a65-bffe-77990ab43428 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:40.533214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535c6bb-0d81-4a65-bffe-77990ab43428 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:40.590618 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270f13a-0c86-47f5-a3cc-34174e2a3b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:40.591290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270f13a-0c86-47f5-a3cc-34174e2a3b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:40.591571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270f13a-0c86-47f5-a3cc-34174e2a3b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:40.591842 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270f13a-0c86-47f5-a3cc-34174e2a3b50 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:40.592527 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270f13a-0c86-47f5-a3cc-34174e2a3b50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:40.594469 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270f13a-0c86-47f5-a3cc-34174e2a3b50 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:40.614469 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1119/2221] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:40 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:40.620241 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905df1cc-40e6-45a0-9686-dced2b59ae66 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:40.630592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905df1cc-40e6-45a0-9686-dced2b59ae66 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:40.635202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bd06e-bd35-4c62-a436-5fdc6a800bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:40.635858 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bd06e-bd35-4c62-a436-5fdc6a800bec admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:40.636138 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bd06e-bd35-4c62-a436-5fdc6a800bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:40.636411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4bd06e-bd35-4c62-a436-5fdc6a800bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:40.637194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4bd06e-bd35-4c62-a436-5fdc6a800bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:40.638787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4bd06e-bd35-4c62-a436-5fdc6a800bec admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:40.649941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1103/2222] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:40.654749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270f13a-0c86-47f5-a3cc-34174e2a3b50 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:40.666559 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270f13a-0c86-47f5-a3cc-34174e2a3b50 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:40.671293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928dcd6-f507-4d55-bb17-eda664d4457f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:40.672201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928dcd6-f507-4d55-bb17-eda664d4457f admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:40.672485 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928dcd6-f507-4d55-bb17-eda664d4457f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:40.672754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928dcd6-f507-4d55-bb17-eda664d4457f admin admin] PATH_INFO: `/v3/projects/d93510ff5ce7435ea0068b5602452a4c/users/9818f0a0f9f34d9693c26312824274ae/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:40.684540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:40.689171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928dcd6-f507-4d55-bb17-eda664d4457f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d93510ff5ce7435ea0068b5602452a4c, user_id=9818f0a0f9f34d9693c26312824274ae, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:40.692017 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928dcd6-f507-4d55-bb17-eda664d4457f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:40.707010 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1120/2223] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:40 2022] PUT /identity/v3/projects/d93510ff5ce7435ea0068b5602452a4c/users/9818f0a0f9f34d9693c26312824274ae/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:40.711481 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bd06e-bd35-4c62-a436-5fdc6a800bec admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:40.723011 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4bd06e-bd35-4c62-a436-5fdc6a800bec admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:40.793889 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59408ca-fbc8-4c59-a3b0-8431b7f76fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:40.794570 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59408ca-fbc8-4c59-a3b0-8431b7f76fdf admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:40.794853 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59408ca-fbc8-4c59-a3b0-8431b7f76fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:40.795125 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59408ca-fbc8-4c59-a3b0-8431b7f76fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:40.795860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59408ca-fbc8-4c59-a3b0-8431b7f76fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:40.797500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59408ca-fbc8-4c59-a3b0-8431b7f76fdf admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:40.809753 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1104/2224] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:40.813972 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928dcd6-f507-4d55-bb17-eda664d4457f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:40.825665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928dcd6-f507-4d55-bb17-eda664d4457f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:40.830828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebf6c6-befb-4b3f-beac-349bc788b46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:40.831730 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebf6c6-befb-4b3f-beac-349bc788b46e admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:40.832065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebf6c6-befb-4b3f-beac-349bc788b46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:40.832345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ebf6c6-befb-4b3f-beac-349bc788b46e admin admin] PATH_INFO: `/v3/projects/d93510ff5ce7435ea0068b5602452a4c/users/9818f0a0f9f34d9693c26312824274ae/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:40.835368 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ebf6c6-befb-4b3f-beac-349bc788b46e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d93510ff5ce7435ea0068b5602452a4c, user_id=9818f0a0f9f34d9693c26312824274ae, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:40.838054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ebf6c6-befb-4b3f-beac-349bc788b46e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:40.848714 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1121/2225] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:40 2022] PUT /identity/v3/projects/d93510ff5ce7435ea0068b5602452a4c/users/9818f0a0f9f34d9693c26312824274ae/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:40.859977 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08636bd9-c0ae-40b6-880d-d1bdf21845ee None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:40.860264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08636bd9-c0ae-40b6-880d-d1bdf21845ee None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:40.860539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08636bd9-c0ae-40b6-880d-d1bdf21845ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:40.888534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-08636bd9-c0ae-40b6-880d-d1bdf21845ee None None] MFA Rules not processed for user `9818f0a0f9f34d9693c26312824274ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:40.890120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-08636bd9-c0ae-40b6-880d-d1bdf21845ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:40.923467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:40.971406 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1105/2226] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:40 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:40.982867 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebf6c6-befb-4b3f-beac-349bc788b46e admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:40.984880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b8ebf6c6-befb-4b3f-beac-349bc788b46e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:41.057190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ebf6c6-befb-4b3f-beac-349bc788b46e admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:41.061194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f98e4-505c-4952-a867-4633d4cb039c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:41.061840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f98e4-505c-4952-a867-4633d4cb039c service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:41.062122 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f98e4-505c-4952-a867-4633d4cb039c service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:41.062419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f98e4-505c-4952-a867-4633d4cb039c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:41.126102 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f98e4-505c-4952-a867-4633d4cb039c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:41.127538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06f98e4-505c-4952-a867-4633d4cb039c service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:41.137195 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1122/2227] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:42.118714 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08636bd9-c0ae-40b6-880d-d1bdf21845ee None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:42.120230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-08636bd9-c0ae-40b6-880d-d1bdf21845ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:42.176370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08636bd9-c0ae-40b6-880d-d1bdf21845ee None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:42.180351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ea1c3-63f6-4582-a62c-96868613ff91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:42.181022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ea1c3-63f6-4582-a62c-96868613ff91 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:42.181306 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ea1c3-63f6-4582-a62c-96868613ff91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:42.181581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ea1c3-63f6-4582-a62c-96868613ff91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:42.183338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b18ea1c3-63f6-4582-a62c-96868613ff91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:42.246831 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ea1c3-63f6-4582-a62c-96868613ff91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:42.248267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ea1c3-63f6-4582-a62c-96868613ff91 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:42.258777 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1106/2228] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:45.469574 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f98e4-505c-4952-a867-4633d4cb039c service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:45.479059 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06f98e4-505c-4952-a867-4633d4cb039c service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:45.482661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a53923-2631-4559-8787-33ccc231f1b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:45.483300 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a53923-2631-4559-8787-33ccc231f1b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:45.483581 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a53923-2631-4559-8787-33ccc231f1b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:45.483854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a53923-2631-4559-8787-33ccc231f1b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:45.493555 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a53923-2631-4559-8787-33ccc231f1b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:45.494972 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a53923-2631-4559-8787-33ccc231f1b3 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:45.504478 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1123/2229] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:45 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:50.462449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ea1c3-63f6-4582-a62c-96868613ff91 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:50.539070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ea1c3-63f6-4582-a62c-96868613ff91 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:50.543745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e45156-f896-44d0-9d71-e4bd56155621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:50.544609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e45156-f896-44d0-9d71-e4bd56155621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:50.544889 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e45156-f896-44d0-9d71-e4bd56155621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:50.545215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e45156-f896-44d0-9d71-e4bd56155621 admin admin] PATH_INFO: `/v3/users/ad16f4273b0b42ad8c362226cbde84c5` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:50.546632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e45156-f896-44d0-9d71-e4bd56155621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad16f4273b0b42ad8c362226cbde84c5)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:50.548901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e45156-f896-44d0-9d71-e4bd56155621 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:50.570123 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:50.594801 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-02e45156-f896-44d0-9d71-e4bd56155621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad16f4273b0b42ad8c362226cbde84c5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:50.595457 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-02e45156-f896-44d0-9d71-e4bd56155621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad16f4273b0b42ad8c362226cbde84c5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:50.607194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-02e45156-f896-44d0-9d71-e4bd56155621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad16f4273b0b42ad8c362226cbde84c5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:50.614695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-02e45156-f896-44d0-9d71-e4bd56155621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad16f4273b0b42ad8c362226cbde84c5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:50.625411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-02e45156-f896-44d0-9d71-e4bd56155621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad16f4273b0b42ad8c362226cbde84c5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:50.631803 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1107/2230] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:50 2022] DELETE /identity/v3/users/ad16f4273b0b42ad8c362226cbde84c5 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:50.636881 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c5d6d-a285-4159-b039-3b0ddebb65ec None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:50.637364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c5d6d-a285-4159-b039-3b0ddebb65ec None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:50.637471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0c5d6d-a285-4159-b039-3b0ddebb65ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:50.655536 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2e0c5d6d-a285-4159-b039-3b0ddebb65ec None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:50.656730 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2e0c5d6d-a285-4159-b039-3b0ddebb65ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:50.714210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1124/2231] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:50.723740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e45156-f896-44d0-9d71-e4bd56155621 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:50.725170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-02e45156-f896-44d0-9d71-e4bd56155621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:50.784159 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e45156-f896-44d0-9d71-e4bd56155621 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:50.787907 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4296d5c-57f1-4fde-98fd-7c19385c1549 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:50.788552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4296d5c-57f1-4fde-98fd-7c19385c1549 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:50.788830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4296d5c-57f1-4fde-98fd-7c19385c1549 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:50.789133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4296d5c-57f1-4fde-98fd-7c19385c1549 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:50.798066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4296d5c-57f1-4fde-98fd-7c19385c1549 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:50.799494 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4296d5c-57f1-4fde-98fd-7c19385c1549 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:50.808756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1108/2232] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:51.095741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c5d6d-a285-4159-b039-3b0ddebb65ec None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:51.097806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2e0c5d6d-a285-4159-b039-3b0ddebb65ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:51.113077 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0c5d6d-a285-4159-b039-3b0ddebb65ec None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:51.121343 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28017ad2-429a-487c-b5b2-0f23b7a5213a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:51.121343 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28017ad2-429a-487c-b5b2-0f23b7a5213a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:51.121343 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28017ad2-429a-487c-b5b2-0f23b7a5213a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:51.121343 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28017ad2-429a-487c-b5b2-0f23b7a5213a admin admin] PATH_INFO: `/v3/projects/7f452e62b7494a3da6953db48cc14969` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:51.122563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28017ad2-429a-487c-b5b2-0f23b7a5213a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f452e62b7494a3da6953db48cc14969)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:51.125048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28017ad2-429a-487c-b5b2-0f23b7a5213a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:51.140377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:51.182746 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-28017ad2-429a-487c-b5b2-0f23b7a5213a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f452e62b7494a3da6953db48cc14969', 'request_id': 'req-28017ad2-429a-487c-b5b2-0f23b7a5213a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:51.189797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-28017ad2-429a-487c-b5b2-0f23b7a5213a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f452e62b7494a3da6953db48cc14969', 'request_id': 'req-28017ad2-429a-487c-b5b2-0f23b7a5213a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:51.196768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-28017ad2-429a-487c-b5b2-0f23b7a5213a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f452e62b7494a3da6953db48cc14969', 'request_id': 'req-28017ad2-429a-487c-b5b2-0f23b7a5213a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:51.209398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-28017ad2-429a-487c-b5b2-0f23b7a5213a admin admin] The token cache is being invalidate because project 7f452e62b7494a3da6953db48cc14969 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:51.209681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-28017ad2-429a-487c-b5b2-0f23b7a5213a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:51.210328 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-28017ad2-429a-487c-b5b2-0f23b7a5213a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:51.211885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1125/2233] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:51 2022] DELETE /identity/v3/projects/7f452e62b7494a3da6953db48cc14969 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:51.215990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4296d5c-57f1-4fde-98fd-7c19385c1549 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:51.226881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4296d5c-57f1-4fde-98fd-7c19385c1549 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:51.295957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f2c6a-3421-4a19-bd3d-45631abcef46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:51.296867 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f2c6a-3421-4a19-bd3d-45631abcef46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:51.297177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f2c6a-3421-4a19-bd3d-45631abcef46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:51.297447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f2c6a-3421-4a19-bd3d-45631abcef46 admin admin] PATH_INFO: `/v3/users/9818f0a0f9f34d9693c26312824274ae` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:51.298658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f2c6a-3421-4a19-bd3d-45631abcef46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9818f0a0f9f34d9693c26312824274ae)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:51.300671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f2c6a-3421-4a19-bd3d-45631abcef46 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:51.349997 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:51.358750 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f83f2c6a-3421-4a19-bd3d-45631abcef46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9818f0a0f9f34d9693c26312824274ae'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:51.360822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f83f2c6a-3421-4a19-bd3d-45631abcef46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9818f0a0f9f34d9693c26312824274ae'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:51.392869 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f83f2c6a-3421-4a19-bd3d-45631abcef46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9818f0a0f9f34d9693c26312824274ae'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:51.401602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f83f2c6a-3421-4a19-bd3d-45631abcef46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9818f0a0f9f34d9693c26312824274ae'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:51.412491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-f83f2c6a-3421-4a19-bd3d-45631abcef46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9818f0a0f9f34d9693c26312824274ae'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:51.424132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1109/2234] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:51 2022] DELETE /identity/v3/users/9818f0a0f9f34d9693c26312824274ae => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:51.723769 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28017ad2-429a-487c-b5b2-0f23b7a5213a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:51.725411 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-28017ad2-429a-487c-b5b2-0f23b7a5213a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:51.746582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28017ad2-429a-487c-b5b2-0f23b7a5213a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:51.805796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f5da4-d1e7-4425-9ab1-076d75b27b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:51.806658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f5da4-d1e7-4425-9ab1-076d75b27b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:51.806939 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f5da4-d1e7-4425-9ab1-076d75b27b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:51.807214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f5da4-d1e7-4425-9ab1-076d75b27b82 admin admin] PATH_INFO: `/v3/projects/d93510ff5ce7435ea0068b5602452a4c` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:51.808389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f5da4-d1e7-4425-9ab1-076d75b27b82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d93510ff5ce7435ea0068b5602452a4c)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:51.810358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f5da4-d1e7-4425-9ab1-076d75b27b82 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:51.861184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-182f5da4-d1e7-4425-9ab1-076d75b27b82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd93510ff5ce7435ea0068b5602452a4c', 'request_id': 'req-182f5da4-d1e7-4425-9ab1-076d75b27b82'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:51.867590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-182f5da4-d1e7-4425-9ab1-076d75b27b82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd93510ff5ce7435ea0068b5602452a4c', 'request_id': 'req-182f5da4-d1e7-4425-9ab1-076d75b27b82'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:51.873350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-182f5da4-d1e7-4425-9ab1-076d75b27b82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd93510ff5ce7435ea0068b5602452a4c', 'request_id': 'req-182f5da4-d1e7-4425-9ab1-076d75b27b82'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:51.886700 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-182f5da4-d1e7-4425-9ab1-076d75b27b82 admin admin] The token cache is being invalidate because project d93510ff5ce7435ea0068b5602452a4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:51.886993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-182f5da4-d1e7-4425-9ab1-076d75b27b82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:51.887692 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-182f5da4-d1e7-4425-9ab1-076d75b27b82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:51.889480 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1126/2235] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:51 2022] DELETE /identity/v3/projects/d93510ff5ce7435ea0068b5602452a4c => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:51.895458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedde88-9ffc-4aab-afae-87cbdc163b50 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:51.895896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedde88-9ffc-4aab-afae-87cbdc163b50 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:51.896172 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcedde88-9ffc-4aab-afae-87cbdc163b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:51.929029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-fcedde88-9ffc-4aab-afae-87cbdc163b50 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:51.930264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fcedde88-9ffc-4aab-afae-87cbdc163b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:51.995398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1110/2236] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:52.004602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b8fd0-3fd0-4e6e-9155-c236e833a8b0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:52.004895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b8fd0-3fd0-4e6e-9155-c236e833a8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:52.005204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b8fd0-3fd0-4e6e-9155-c236e833a8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:52.024821 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ce0b8fd0-3fd0-4e6e-9155-c236e833a8b0 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:52.026074 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ce0b8fd0-3fd0-4e6e-9155-c236e833a8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:52.040606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1127/2237] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:52.044964 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcedde88-9ffc-4aab-afae-87cbdc163b50 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:52.055744 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcedde88-9ffc-4aab-afae-87cbdc163b50 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:52.060357 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d0e66-89cd-4651-863c-cb9b9c70f2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:52.061051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d0e66-89cd-4651-863c-cb9b9c70f2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:52.061336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d0e66-89cd-4651-863c-cb9b9c70f2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:52.061612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09d0e66-89cd-4651-863c-cb9b9c70f2c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:52.062355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d0e66-89cd-4651-863c-cb9b9c70f2c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:15:52.062662 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d0e66-89cd-4651-863c-cb9b9c70f2c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:52.064432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09d0e66-89cd-4651-863c-cb9b9c70f2c3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:52.078681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1111/2238] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:15:52.083284 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b8fd0-3fd0-4e6e-9155-c236e833a8b0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:52.093106 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b8fd0-3fd0-4e6e-9155-c236e833a8b0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:52.097847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55236be-3c42-4cd7-8532-d53cf15bdf2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:52.098503 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55236be-3c42-4cd7-8532-d53cf15bdf2f admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:52.098784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55236be-3c42-4cd7-8532-d53cf15bdf2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:52.099155 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55236be-3c42-4cd7-8532-d53cf15bdf2f admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:52.099749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55236be-3c42-4cd7-8532-d53cf15bdf2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:52.101718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55236be-3c42-4cd7-8532-d53cf15bdf2f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:52.127283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1128/2239] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:52 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:52.131586 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d0e66-89cd-4651-863c-cb9b9c70f2c3 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:52.142024 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09d0e66-89cd-4651-863c-cb9b9c70f2c3 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:52.212709 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62b2af-fffa-4f37-9d77-bb4062d4f44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:52.213375 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62b2af-fffa-4f37-9d77-bb4062d4f44a admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:52.213656 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62b2af-fffa-4f37-9d77-bb4062d4f44a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:52.214005 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62b2af-fffa-4f37-9d77-bb4062d4f44a admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:52.214701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62b2af-fffa-4f37-9d77-bb4062d4f44a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:52.216741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62b2af-fffa-4f37-9d77-bb4062d4f44a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:52.238460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1112/2240] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:52 2022] POST /identity/v3/users => generated 287 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:52.243064 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55236be-3c42-4cd7-8532-d53cf15bdf2f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:52.255201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:52.265091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55236be-3c42-4cd7-8532-d53cf15bdf2f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:52.270665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf51777-3b41-424c-879b-c62fe0fa2711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:52.271337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf51777-3b41-424c-879b-c62fe0fa2711 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:52.271619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf51777-3b41-424c-879b-c62fe0fa2711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:52.271896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf51777-3b41-424c-879b-c62fe0fa2711 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:52.272710 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf51777-3b41-424c-879b-c62fe0fa2711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:52.275132 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf51777-3b41-424c-879b-c62fe0fa2711 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:52.293142 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1129/2241] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:15:52.296452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62b2af-fffa-4f37-9d77-bb4062d4f44a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:52.308306 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62b2af-fffa-4f37-9d77-bb4062d4f44a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:52.314029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa93b9b-13d7-4206-8cb1-05e9985af1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:52.314704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa93b9b-13d7-4206-8cb1-05e9985af1d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:52.314987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa93b9b-13d7-4206-8cb1-05e9985af1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:52.315262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa93b9b-13d7-4206-8cb1-05e9985af1d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:52.315999 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa93b9b-13d7-4206-8cb1-05e9985af1d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:52.317821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa93b9b-13d7-4206-8cb1-05e9985af1d1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:52.329351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1113/2242] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:52.333517 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf51777-3b41-424c-879b-c62fe0fa2711 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:52.345554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf51777-3b41-424c-879b-c62fe0fa2711 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:52.351549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7d2bb-d4b8-427b-9610-fef18db33466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:52.352447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7d2bb-d4b8-427b-9610-fef18db33466 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:52.352850 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7d2bb-d4b8-427b-9610-fef18db33466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:52.353126 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7d2bb-d4b8-427b-9610-fef18db33466 admin admin] PATH_INFO: `/v3/projects/197216f9baf547afa7dd5605f4482dc6/users/ebcaf4bd9249408eb6038c85c25e644d/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:52.371372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7d2bb-d4b8-427b-9610-fef18db33466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=197216f9baf547afa7dd5605f4482dc6, user_id=ebcaf4bd9249408eb6038c85c25e644d, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:52.374541 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7d2bb-d4b8-427b-9610-fef18db33466 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:52.389208 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1130/2243] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:52 2022] PUT /identity/v3/projects/197216f9baf547afa7dd5605f4482dc6/users/ebcaf4bd9249408eb6038c85c25e644d/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:15:52.400543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e7abb-67b0-421d-b2f7-99016425a33c None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:52.400835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e7abb-67b0-421d-b2f7-99016425a33c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:52.401151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e7abb-67b0-421d-b2f7-99016425a33c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:52.427326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9a1e7abb-67b0-421d-b2f7-99016425a33c None None] MFA Rules not processed for user `ebcaf4bd9249408eb6038c85c25e644d`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:52.428542 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9a1e7abb-67b0-421d-b2f7-99016425a33c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:52.441487 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:52.524655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1114/2244] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:52.536148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7d2bb-d4b8-427b-9610-fef18db33466 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:52.538034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8ba7d2bb-d4b8-427b-9610-fef18db33466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:52.601421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7d2bb-d4b8-427b-9610-fef18db33466 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:52.605246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28d9d0b-a940-42fb-a989-ad3ff42ef658 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:52.605890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d9d0b-a940-42fb-a989-ad3ff42ef658 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:52.606764 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d9d0b-a940-42fb-a989-ad3ff42ef658 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:52.607084 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28d9d0b-a940-42fb-a989-ad3ff42ef658 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:52.622021 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d9d0b-a940-42fb-a989-ad3ff42ef658 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:52.623468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28d9d0b-a940-42fb-a989-ad3ff42ef658 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:52.633599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1131/2245] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:52.653115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e7abb-67b0-421d-b2f7-99016425a33c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:52.725421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e7abb-67b0-421d-b2f7-99016425a33c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:52.739367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3424a9-452a-46ec-992c-bf9ca20323f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:52.740335 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3424a9-452a-46ec-992c-bf9ca20323f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:52.740615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3424a9-452a-46ec-992c-bf9ca20323f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:52.740914 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3424a9-452a-46ec-992c-bf9ca20323f0 admin admin] PATH_INFO: `/v3/users/ebcaf4bd9249408eb6038c85c25e644d` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:52.742723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3424a9-452a-46ec-992c-bf9ca20323f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebcaf4bd9249408eb6038c85c25e644d)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:52.744670 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3424a9-452a-46ec-992c-bf9ca20323f0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:52.793121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9c3424a9-452a-46ec-992c-bf9ca20323f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebcaf4bd9249408eb6038c85c25e644d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:52.793707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9c3424a9-452a-46ec-992c-bf9ca20323f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebcaf4bd9249408eb6038c85c25e644d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:52.806544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9c3424a9-452a-46ec-992c-bf9ca20323f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebcaf4bd9249408eb6038c85c25e644d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:52.813456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9c3424a9-452a-46ec-992c-bf9ca20323f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebcaf4bd9249408eb6038c85c25e644d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:52.821044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9c3424a9-452a-46ec-992c-bf9ca20323f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebcaf4bd9249408eb6038c85c25e644d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:52.826914 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1115/2246] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:15:52 2022] DELETE /identity/v3/users/ebcaf4bd9249408eb6038c85c25e644d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:52.831380 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708524b-3af4-4f05-b4ed-9ab48c208938 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:52.831831 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708524b-3af4-4f05-b4ed-9ab48c208938 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:52.831941 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8708524b-3af4-4f05-b4ed-9ab48c208938 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:52.848891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8708524b-3af4-4f05-b4ed-9ab48c208938 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:52.850110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8708524b-3af4-4f05-b4ed-9ab48c208938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:52.908642 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1132/2247] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:52.918850 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3424a9-452a-46ec-992c-bf9ca20323f0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:52.920235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9c3424a9-452a-46ec-992c-bf9ca20323f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:52.957668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:52.983272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3424a9-452a-46ec-992c-bf9ca20323f0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:52.988385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9682e6-db9b-4ebb-ad68-21f0d6936cf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:52.989479 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9682e6-db9b-4ebb-ad68-21f0d6936cf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:52.989952 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9682e6-db9b-4ebb-ad68-21f0d6936cf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:52.990438 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9682e6-db9b-4ebb-ad68-21f0d6936cf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:53.000366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9682e6-db9b-4ebb-ad68-21f0d6936cf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:53.001820 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9682e6-db9b-4ebb-ad68-21f0d6936cf9 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:53.011244 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1116/2248] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:53.293042 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708524b-3af4-4f05-b4ed-9ab48c208938 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:53.294497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8708524b-3af4-4f05-b4ed-9ab48c208938 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:53.306962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8708524b-3af4-4f05-b4ed-9ab48c208938 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:53.311729 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54153a17-8b8a-47f2-9f52-677be5ac7229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:53.312628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54153a17-8b8a-47f2-9f52-677be5ac7229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:53.312908 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54153a17-8b8a-47f2-9f52-677be5ac7229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:53.313223 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54153a17-8b8a-47f2-9f52-677be5ac7229 admin admin] PATH_INFO: `/v3/projects/197216f9baf547afa7dd5605f4482dc6` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:53.314462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54153a17-8b8a-47f2-9f52-677be5ac7229 admin admin] RBAC: Authorizing `identity:delete_project(project_id=197216f9baf547afa7dd5605f4482dc6)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:53.316391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54153a17-8b8a-47f2-9f52-677be5ac7229 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:53.364605 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-54153a17-8b8a-47f2-9f52-677be5ac7229 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '197216f9baf547afa7dd5605f4482dc6', 'request_id': 'req-54153a17-8b8a-47f2-9f52-677be5ac7229'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:53.370754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-54153a17-8b8a-47f2-9f52-677be5ac7229 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '197216f9baf547afa7dd5605f4482dc6', 'request_id': 'req-54153a17-8b8a-47f2-9f52-677be5ac7229'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:53.376149 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-54153a17-8b8a-47f2-9f52-677be5ac7229 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '197216f9baf547afa7dd5605f4482dc6', 'request_id': 'req-54153a17-8b8a-47f2-9f52-677be5ac7229'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:53.378587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:53.390050 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-54153a17-8b8a-47f2-9f52-677be5ac7229 admin admin] The token cache is being invalidate because project 197216f9baf547afa7dd5605f4482dc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:15:53.390336 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-54153a17-8b8a-47f2-9f52-677be5ac7229 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:53.390977 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-54153a17-8b8a-47f2-9f52-677be5ac7229 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:15:53.392541 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1133/2249] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:15:53 2022] DELETE /identity/v3/projects/197216f9baf547afa7dd5605f4482dc6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:53.398127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d78a0-4d21-4047-9a35-0cebd7f08073 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:53.398414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d78a0-4d21-4047-9a35-0cebd7f08073 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:53.398685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49d78a0-4d21-4047-9a35-0cebd7f08073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:53.417055 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f49d78a0-4d21-4047-9a35-0cebd7f08073 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:53.418287 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f49d78a0-4d21-4047-9a35-0cebd7f08073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:53.480190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1117/2250] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:53.488162 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcc9bb-129c-4fa5-9415-1ea7c17bec07 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:53.489303 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcc9bb-129c-4fa5-9415-1ea7c17bec07 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:53.489592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcc9bb-129c-4fa5-9415-1ea7c17bec07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:53.509993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f8bcc9bb-129c-4fa5-9415-1ea7c17bec07 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:53.511206 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f8bcc9bb-129c-4fa5-9415-1ea7c17bec07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:53.519151 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1134/2251] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:53.524032 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d78a0-4d21-4047-9a35-0cebd7f08073 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:53.533915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49d78a0-4d21-4047-9a35-0cebd7f08073 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:53.538644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a3aab-79f7-42bd-9ee9-e9570a30ee5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:53.539308 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a3aab-79f7-42bd-9ee9-e9570a30ee5b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:53.539680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a3aab-79f7-42bd-9ee9-e9570a30ee5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:53.539962 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a3aab-79f7-42bd-9ee9-e9570a30ee5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:53.540703 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a3aab-79f7-42bd-9ee9-e9570a30ee5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:15:53.541042 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a3aab-79f7-42bd-9ee9-e9570a30ee5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:53.542637 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a3aab-79f7-42bd-9ee9-e9570a30ee5b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:53.554480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1118/2252] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:15:53.558736 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcc9bb-129c-4fa5-9415-1ea7c17bec07 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:53.567911 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcc9bb-129c-4fa5-9415-1ea7c17bec07 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:53.573264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f6930-e5b0-49cc-97e8-78d3879a3c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:53.573924 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f6930-e5b0-49cc-97e8-78d3879a3c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:53.574206 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f6930-e5b0-49cc-97e8-78d3879a3c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:53.574603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f6930-e5b0-49cc-97e8-78d3879a3c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:53.575302 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f6930-e5b0-49cc-97e8-78d3879a3c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:53.577256 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f6930-e5b0-49cc-97e8-78d3879a3c3a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:53.606432 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1135/2253] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:53 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:53.610528 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a3aab-79f7-42bd-9ee9-e9570a30ee5b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:53.620536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a3aab-79f7-42bd-9ee9-e9570a30ee5b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:53.680345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc52ca5-4fb6-405e-af4c-7d05b8f8a788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:53.681254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc52ca5-4fb6-405e-af4c-7d05b8f8a788 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:53.681590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc52ca5-4fb6-405e-af4c-7d05b8f8a788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:53.681864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc52ca5-4fb6-405e-af4c-7d05b8f8a788 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:53.682578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc52ca5-4fb6-405e-af4c-7d05b8f8a788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:53.684497 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc52ca5-4fb6-405e-af4c-7d05b8f8a788 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:53.705428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1119/2254] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:53 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:53.709900 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f6930-e5b0-49cc-97e8-78d3879a3c3a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:53.720561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f6930-e5b0-49cc-97e8-78d3879a3c3a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:53.725073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a603a3-402a-4f3d-a777-b1bc6de54a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:53.725770 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a603a3-402a-4f3d-a777-b1bc6de54a0d admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:53.726046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a603a3-402a-4f3d-a777-b1bc6de54a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:53.726310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a603a3-402a-4f3d-a777-b1bc6de54a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:53.727029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a603a3-402a-4f3d-a777-b1bc6de54a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:53.728603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a603a3-402a-4f3d-a777-b1bc6de54a0d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:53.738759 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1136/2255] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:53.742317 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc52ca5-4fb6-405e-af4c-7d05b8f8a788 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:53.752372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc52ca5-4fb6-405e-af4c-7d05b8f8a788 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:53.757547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0da934-5684-4e16-9f23-0a85fc6a7a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:53.758202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0da934-5684-4e16-9f23-0a85fc6a7a8b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:53.758481 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0da934-5684-4e16-9f23-0a85fc6a7a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:53.758749 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0da934-5684-4e16-9f23-0a85fc6a7a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:53.759473 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0da934-5684-4e16-9f23-0a85fc6a7a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:53.761087 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0da934-5684-4e16-9f23-0a85fc6a7a8b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:53.770968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1120/2256] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:53.775719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a603a3-402a-4f3d-a777-b1bc6de54a0d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:53.785893 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a603a3-402a-4f3d-a777-b1bc6de54a0d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:53.791372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076e17d-ffa8-4445-88c3-ae5088839187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:53.792960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076e17d-ffa8-4445-88c3-ae5088839187 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:53.793252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076e17d-ffa8-4445-88c3-ae5088839187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:53.793580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9076e17d-ffa8-4445-88c3-ae5088839187 admin admin] PATH_INFO: `/v3/projects/c1988391f9a5459ab28b25aec6072d42/users/0066522fd3014e63847bfc624efcdeaf/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:53.811120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076e17d-ffa8-4445-88c3-ae5088839187 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1988391f9a5459ab28b25aec6072d42, user_id=0066522fd3014e63847bfc624efcdeaf, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:53.813747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9076e17d-ffa8-4445-88c3-ae5088839187 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:53.833242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1137/2257] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:53 2022] PUT /identity/v3/projects/c1988391f9a5459ab28b25aec6072d42/users/0066522fd3014e63847bfc624efcdeaf/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:53.844646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd27cd5-0f86-4e05-a018-079205bf437a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:53.845128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd27cd5-0f86-4e05-a018-079205bf437a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:53.845417 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd27cd5-0f86-4e05-a018-079205bf437a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:53.876368 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-2dd27cd5-0f86-4e05-a018-079205bf437a None None] MFA Rules not processed for user `0066522fd3014e63847bfc624efcdeaf`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:53.877438 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2dd27cd5-0f86-4e05-a018-079205bf437a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:53.918327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:53.965518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1121/2258] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:53.969912 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076e17d-ffa8-4445-88c3-ae5088839187 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:54.035861 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9076e17d-ffa8-4445-88c3-ae5088839187 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:54.041067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb97d71-75dc-4703-9d40-e9c1c67693f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:54.041900 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb97d71-75dc-4703-9d40-e9c1c67693f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:54.042178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb97d71-75dc-4703-9d40-e9c1c67693f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:54.042454 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb97d71-75dc-4703-9d40-e9c1c67693f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:54.043141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb97d71-75dc-4703-9d40-e9c1c67693f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:54.045038 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb97d71-75dc-4703-9d40-e9c1c67693f8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:54.078053 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1138/2259] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:53 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:54.082101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd27cd5-0f86-4e05-a018-079205bf437a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:54.093622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd27cd5-0f86-4e05-a018-079205bf437a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:54.170697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2b01b-60f2-4249-8d18-d03b68d121ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:54.171370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2b01b-60f2-4249-8d18-d03b68d121ad admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:54.171649 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2b01b-60f2-4249-8d18-d03b68d121ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:54.171925 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2b01b-60f2-4249-8d18-d03b68d121ad admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:54.172630 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2b01b-60f2-4249-8d18-d03b68d121ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:54.174648 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a2b01b-60f2-4249-8d18-d03b68d121ad admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:54.196238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1122/2260] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:54 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:54.200526 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb97d71-75dc-4703-9d40-e9c1c67693f8 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:54.210714 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb97d71-75dc-4703-9d40-e9c1c67693f8 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:54.215329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afebe3-c0f9-47a2-957e-4168445ec1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:54.216112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afebe3-c0f9-47a2-957e-4168445ec1d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:54.216278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afebe3-c0f9-47a2-957e-4168445ec1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:54.216569 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afebe3-c0f9-47a2-957e-4168445ec1d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:54.217599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afebe3-c0f9-47a2-957e-4168445ec1d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:54.219179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afebe3-c0f9-47a2-957e-4168445ec1d6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:54.233487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1139/2261] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:54.238388 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2b01b-60f2-4249-8d18-d03b68d121ad admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:54.250590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2b01b-60f2-4249-8d18-d03b68d121ad admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:54.254715 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838df84-d294-4937-b3ad-87e18328bfee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:54.255383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838df84-d294-4937-b3ad-87e18328bfee admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:54.255663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838df84-d294-4937-b3ad-87e18328bfee admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:54.255936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838df84-d294-4937-b3ad-87e18328bfee admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:54.256670 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838df84-d294-4937-b3ad-87e18328bfee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:54.258747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838df84-d294-4937-b3ad-87e18328bfee admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:54.269229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1123/2262] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:54.273421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afebe3-c0f9-47a2-957e-4168445ec1d6 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:54.284251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afebe3-c0f9-47a2-957e-4168445ec1d6 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:54.290445 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d3917-1748-42c1-9953-78f1de616abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:54.291313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d3917-1748-42c1-9953-78f1de616abd admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:54.291711 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d3917-1748-42c1-9953-78f1de616abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:54.291983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2d3917-1748-42c1-9953-78f1de616abd admin admin] PATH_INFO: `/v3/projects/df5f885b55e84739bca9e718f2a95a84/users/82c31197ddc14ffe8bad09f93002affe/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:54.307861 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d3917-1748-42c1-9953-78f1de616abd admin admin] RBAC: Authorizing `identity:create_grant(project_id=df5f885b55e84739bca9e718f2a95a84, user_id=82c31197ddc14ffe8bad09f93002affe, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:54.310659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2d3917-1748-42c1-9953-78f1de616abd admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:54.336725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1140/2263] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:54 2022] PUT /identity/v3/projects/df5f885b55e84739bca9e718f2a95a84/users/82c31197ddc14ffe8bad09f93002affe/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:54.346658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d37ba-9502-43f5-b69f-39d10912efc9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:54.346940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d37ba-9502-43f5-b69f-39d10912efc9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:54.347211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d37ba-9502-43f5-b69f-39d10912efc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:54.375692 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c54d37ba-9502-43f5-b69f-39d10912efc9 None None] MFA Rules not processed for user `82c31197ddc14ffe8bad09f93002affe`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:54.376895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c54d37ba-9502-43f5-b69f-39d10912efc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:54.460973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1124/2264] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:54.462983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d3917-1748-42c1-9953-78f1de616abd admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:54.518711 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:54.557309 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2d3917-1748-42c1-9953-78f1de616abd admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:54.562256 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f037a08-b844-4732-addb-af0b64dbb970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:54.563121 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f037a08-b844-4732-addb-af0b64dbb970 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:54.563337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f037a08-b844-4732-addb-af0b64dbb970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:54.563616 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f037a08-b844-4732-addb-af0b64dbb970 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:54.564317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f037a08-b844-4732-addb-af0b64dbb970 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:54.567219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f037a08-b844-4732-addb-af0b64dbb970 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:54.610547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1141/2265] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:15:54 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:15:54.612515 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d37ba-9502-43f5-b69f-39d10912efc9 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:54.630231 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d37ba-9502-43f5-b69f-39d10912efc9 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:54.697355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:15:54.732871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a03845-071b-415d-b0fc-db337148eef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:54.733587 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a03845-071b-415d-b0fc-db337148eef5 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:54.733871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a03845-071b-415d-b0fc-db337148eef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:54.734147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a03845-071b-415d-b0fc-db337148eef5 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:54.734985 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a03845-071b-415d-b0fc-db337148eef5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:54.736961 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a03845-071b-415d-b0fc-db337148eef5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:54.756787 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1125/2266] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:15:54 2022] POST /identity/v3/users => generated 299 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:15:54.760686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f037a08-b844-4732-addb-af0b64dbb970 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:54.770773 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f037a08-b844-4732-addb-af0b64dbb970 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:54.775867 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04414411-56a0-4484-a41c-eea1d256c4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:54.776517 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04414411-56a0-4484-a41c-eea1d256c4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:54.777069 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04414411-56a0-4484-a41c-eea1d256c4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:54.777361 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04414411-56a0-4484-a41c-eea1d256c4b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:54.778085 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04414411-56a0-4484-a41c-eea1d256c4b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:54.779668 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04414411-56a0-4484-a41c-eea1d256c4b0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:54.790799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1142/2267] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:54.795097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a03845-071b-415d-b0fc-db337148eef5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:54.805129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a03845-071b-415d-b0fc-db337148eef5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:54.809745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5c024-8b37-4ac1-846e-b77bc3406cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:54.810614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5c024-8b37-4ac1-846e-b77bc3406cee admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:54.810892 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5c024-8b37-4ac1-846e-b77bc3406cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:54.811159 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb5c024-8b37-4ac1-846e-b77bc3406cee admin admin] PATH_INFO: `/v3/projects/f76fb6e70226460590f79ad0746967ca/users/2c66259ee6604f689f489bd891d449a6/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:54.825551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5c024-8b37-4ac1-846e-b77bc3406cee admin admin] RBAC: Authorizing `identity:create_grant(project_id=f76fb6e70226460590f79ad0746967ca, user_id=2c66259ee6604f689f489bd891d449a6, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:54.828129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb5c024-8b37-4ac1-846e-b77bc3406cee admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:54.841325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1126/2268] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:54 2022] PUT /identity/v3/projects/f76fb6e70226460590f79ad0746967ca/users/2c66259ee6604f689f489bd891d449a6/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:54.845164 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04414411-56a0-4484-a41c-eea1d256c4b0 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:54.855772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04414411-56a0-4484-a41c-eea1d256c4b0 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:54.911005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17275e57-4af8-4bfc-a940-5081b2a53a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:54.911659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17275e57-4af8-4bfc-a940-5081b2a53a7f admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:54.911936 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17275e57-4af8-4bfc-a940-5081b2a53a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:54.912204 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17275e57-4af8-4bfc-a940-5081b2a53a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:54.912916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17275e57-4af8-4bfc-a940-5081b2a53a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:54.915143 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17275e57-4af8-4bfc-a940-5081b2a53a7f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:54.924465 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1143/2269] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:15:54.928552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5c024-8b37-4ac1-846e-b77bc3406cee admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:54.939266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb5c024-8b37-4ac1-846e-b77bc3406cee admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:54.944121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96896830-49b0-46fc-9d42-b9d3a1a1d4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:54.945175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96896830-49b0-46fc-9d42-b9d3a1a1d4b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:54.945457 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96896830-49b0-46fc-9d42-b9d3a1a1d4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:54.945739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96896830-49b0-46fc-9d42-b9d3a1a1d4b7 admin admin] PATH_INFO: `/v3/projects/f76fb6e70226460590f79ad0746967ca/users/2c66259ee6604f689f489bd891d449a6/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:54.947839 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96896830-49b0-46fc-9d42-b9d3a1a1d4b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f76fb6e70226460590f79ad0746967ca, user_id=2c66259ee6604f689f489bd891d449a6, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:54.950540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96896830-49b0-46fc-9d42-b9d3a1a1d4b7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:54.972966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1127/2270] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:15:54 2022] PUT /identity/v3/projects/f76fb6e70226460590f79ad0746967ca/users/2c66259ee6604f689f489bd891d449a6/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:15:54.974451 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb36ee4-d040-4a21-8014-febb5e70c0f3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:54.974732 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb36ee4-d040-4a21-8014-febb5e70c0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:54.975018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb36ee4-d040-4a21-8014-febb5e70c0f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:55.010522 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-fdb36ee4-d040-4a21-8014-febb5e70c0f3 None None] MFA Rules not processed for user `2c66259ee6604f689f489bd891d449a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:15:55.012029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fdb36ee4-d040-4a21-8014-febb5e70c0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:55.089794 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1144/2271] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:15:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:15:55.101180 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96896830-49b0-46fc-9d42-b9d3a1a1d4b7 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:55.102929 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-96896830-49b0-46fc-9d42-b9d3a1a1d4b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:55.160878 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96896830-49b0-46fc-9d42-b9d3a1a1d4b7 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:55.164599 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0c4cba-40ee-4ba2-92a4-b97d03188f7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:55.165269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c4cba-40ee-4ba2-92a4-b97d03188f7d service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:55.165632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c4cba-40ee-4ba2-92a4-b97d03188f7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:55.165913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0c4cba-40ee-4ba2-92a4-b97d03188f7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:15:55.220598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c4cba-40ee-4ba2-92a4-b97d03188f7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:15:55.222125 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0c4cba-40ee-4ba2-92a4-b97d03188f7d service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:15:55.230984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1128/2272] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:15:59.962733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb36ee4-d040-4a21-8014-febb5e70c0f3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:15:59.964105 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fdb36ee4-d040-4a21-8014-febb5e70c0f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:15:59.990733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb36ee4-d040-4a21-8014-febb5e70c0f3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:15:59.995444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dec68c-67ba-466b-906b-c4fa0b05e4a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:15:59.996345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dec68c-67ba-466b-906b-c4fa0b05e4a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:15:59.996633 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dec68c-67ba-466b-906b-c4fa0b05e4a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:15:59.996965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dec68c-67ba-466b-906b-c4fa0b05e4a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:00.032239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:00.057834 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dec68c-67ba-466b-906b-c4fa0b05e4a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:00.059262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dec68c-67ba-466b-906b-c4fa0b05e4a7 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:00.068235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1145/2273] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:15:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:16:00.486600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc866e-9ed8-4e0b-929d-5aefedb97410 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:00.487151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc866e-9ed8-4e0b-929d-5aefedb97410 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:00.487151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc866e-9ed8-4e0b-929d-5aefedb97410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:00.508657 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a8fc866e-9ed8-4e0b-929d-5aefedb97410 None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:00.510062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a8fc866e-9ed8-4e0b-929d-5aefedb97410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:00.558560 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:00.575577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1129/2274] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Jul 14 10:16:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:16:00.627572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dec68c-67ba-466b-906b-c4fa0b05e4a7 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:00.629029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-59dec68c-67ba-466b-906b-c4fa0b05e4a7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:00.713579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc866e-9ed8-4e0b-929d-5aefedb97410 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:00.717857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dec68c-67ba-466b-906b-c4fa0b05e4a7 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:00.722680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc866e-9ed8-4e0b-929d-5aefedb97410 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:00.722910 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e855d6aa-f084-494b-874a-2b57be4127ea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:00.723705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855d6aa-f084-494b-874a-2b57be4127ea service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:00.723989 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855d6aa-f084-494b-874a-2b57be4127ea service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:00.724259 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e855d6aa-f084-494b-874a-2b57be4127ea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:00.727037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89f6ee-3bf9-4512-9044-704fe73cc57a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:00.727668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89f6ee-3bf9-4512-9044-704fe73cc57a service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:00.727946 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89f6ee-3bf9-4512-9044-704fe73cc57a service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:00.728225 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f89f6ee-3bf9-4512-9044-704fe73cc57a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:00.733369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e855d6aa-f084-494b-874a-2b57be4127ea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:00.735239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e855d6aa-f084-494b-874a-2b57be4127ea service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:00.738194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89f6ee-3bf9-4512-9044-704fe73cc57a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:00.739610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f89f6ee-3bf9-4512-9044-704fe73cc57a service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:00.743771 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1146/2275] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:16:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 14 10:16:00.752283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1130/2276] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:16:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:16:04.101901 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c375569-9da9-464e-b962-a1ce30255a8b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:04.102780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c375569-9da9-464e-b962-a1ce30255a8b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:04.102780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c375569-9da9-464e-b962-a1ce30255a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:04.122584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6c375569-9da9-464e-b962-a1ce30255a8b None None] MFA Rules not processed for user `bff5f2d1367545f2b4c77fe0d9f095ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:04.123824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6c375569-9da9-464e-b962-a1ce30255a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:04.185046 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1147/2277] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:16:05.018672 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89f6ee-3bf9-4512-9044-704fe73cc57a service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:05.156032 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f89f6ee-3bf9-4512-9044-704fe73cc57a service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:05.161194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90c9f9-c896-47f9-ac94-cd98d52a5b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:05.162178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90c9f9-c896-47f9-ac94-cd98d52a5b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:05.162469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90c9f9-c896-47f9-ac94-cd98d52a5b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:05.162739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b90c9f9-c896-47f9-ac94-cd98d52a5b3a admin admin] PATH_INFO: `/v3/users/0066522fd3014e63847bfc624efcdeaf` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:05.164503 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b90c9f9-c896-47f9-ac94-cd98d52a5b3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0066522fd3014e63847bfc624efcdeaf)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:05.166469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b90c9f9-c896-47f9-ac94-cd98d52a5b3a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:05.257443 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2b90c9f9-c896-47f9-ac94-cd98d52a5b3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0066522fd3014e63847bfc624efcdeaf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:05.258142 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2b90c9f9-c896-47f9-ac94-cd98d52a5b3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0066522fd3014e63847bfc624efcdeaf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:05.274290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2b90c9f9-c896-47f9-ac94-cd98d52a5b3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0066522fd3014e63847bfc624efcdeaf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:05.283612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2b90c9f9-c896-47f9-ac94-cd98d52a5b3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0066522fd3014e63847bfc624efcdeaf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:05.291646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2b90c9f9-c896-47f9-ac94-cd98d52a5b3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0066522fd3014e63847bfc624efcdeaf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:05.300748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1131/2278] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:16:05 2022] DELETE /identity/v3/users/0066522fd3014e63847bfc624efcdeaf => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:16:05.305904 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748d4f5-3713-4c8d-b802-c0f9abbe7f0c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:05.306818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748d4f5-3713-4c8d-b802-c0f9abbe7f0c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:05.307283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748d4f5-3713-4c8d-b802-c0f9abbe7f0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:05.328555 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-b748d4f5-3713-4c8d-b802-c0f9abbe7f0c None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:05.330112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b748d4f5-3713-4c8d-b802-c0f9abbe7f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:05.412439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1148/2279] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:05.430443 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90c9f9-c896-47f9-ac94-cd98d52a5b3a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:05.435937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2b90c9f9-c896-47f9-ac94-cd98d52a5b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:05.564048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b90c9f9-c896-47f9-ac94-cd98d52a5b3a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:05.570599 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45320485-bbe3-4716-bcc6-acfb6f959feb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:05.571548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45320485-bbe3-4716-bcc6-acfb6f959feb service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:05.571987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45320485-bbe3-4716-bcc6-acfb6f959feb service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:05.572358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45320485-bbe3-4716-bcc6-acfb6f959feb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:05.585930 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45320485-bbe3-4716-bcc6-acfb6f959feb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:05.588272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45320485-bbe3-4716-bcc6-acfb6f959feb service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:05.594623 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:05.600030 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1132/2280] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:16:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:16:06.326072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748d4f5-3713-4c8d-b802-c0f9abbe7f0c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:06.327930 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b748d4f5-3713-4c8d-b802-c0f9abbe7f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:06.359736 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748d4f5-3713-4c8d-b802-c0f9abbe7f0c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:06.385520 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cba8da-df96-4521-ab8c-aa0cf90c17c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:06.388481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cba8da-df96-4521-ab8c-aa0cf90c17c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:06.388738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cba8da-df96-4521-ab8c-aa0cf90c17c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:06.391094 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cba8da-df96-4521-ab8c-aa0cf90c17c0 admin admin] PATH_INFO: `/v3/projects/c1988391f9a5459ab28b25aec6072d42` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:06.393128 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cba8da-df96-4521-ab8c-aa0cf90c17c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1988391f9a5459ab28b25aec6072d42)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:06.397048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cba8da-df96-4521-ab8c-aa0cf90c17c0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:06.401156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:06.495026 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-90cba8da-df96-4521-ab8c-aa0cf90c17c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1988391f9a5459ab28b25aec6072d42', 'request_id': 'req-90cba8da-df96-4521-ab8c-aa0cf90c17c0'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:06.503111 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-90cba8da-df96-4521-ab8c-aa0cf90c17c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1988391f9a5459ab28b25aec6072d42', 'request_id': 'req-90cba8da-df96-4521-ab8c-aa0cf90c17c0'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:06.510346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-90cba8da-df96-4521-ab8c-aa0cf90c17c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1988391f9a5459ab28b25aec6072d42', 'request_id': 'req-90cba8da-df96-4521-ab8c-aa0cf90c17c0'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:06.537597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-90cba8da-df96-4521-ab8c-aa0cf90c17c0 admin admin] The token cache is being invalidate because project c1988391f9a5459ab28b25aec6072d42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:16:06.541333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-90cba8da-df96-4521-ab8c-aa0cf90c17c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:06.542210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-90cba8da-df96-4521-ab8c-aa0cf90c17c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:06.547036 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1149/2281] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:16:06 2022] DELETE /identity/v3/projects/c1988391f9a5459ab28b25aec6072d42 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:16:06.552187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45320485-bbe3-4716-bcc6-acfb6f959feb service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:06.564442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45320485-bbe3-4716-bcc6-acfb6f959feb service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:06.668114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2782f-e8ef-4672-bf2d-c6a1e82ff678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:06.669807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2782f-e8ef-4672-bf2d-c6a1e82ff678 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:06.670278 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2782f-e8ef-4672-bf2d-c6a1e82ff678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:06.670732 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2782f-e8ef-4672-bf2d-c6a1e82ff678 admin admin] PATH_INFO: `/v3/users/82c31197ddc14ffe8bad09f93002affe` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:06.672755 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2782f-e8ef-4672-bf2d-c6a1e82ff678 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82c31197ddc14ffe8bad09f93002affe)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:06.675245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2782f-e8ef-4672-bf2d-c6a1e82ff678 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:06.745762 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2bd2782f-e8ef-4672-bf2d-c6a1e82ff678 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82c31197ddc14ffe8bad09f93002affe'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:06.746505 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2bd2782f-e8ef-4672-bf2d-c6a1e82ff678 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82c31197ddc14ffe8bad09f93002affe'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:06.764001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2bd2782f-e8ef-4672-bf2d-c6a1e82ff678 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82c31197ddc14ffe8bad09f93002affe'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:06.773913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2bd2782f-e8ef-4672-bf2d-c6a1e82ff678 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82c31197ddc14ffe8bad09f93002affe'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:06.789322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2bd2782f-e8ef-4672-bf2d-c6a1e82ff678 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82c31197ddc14ffe8bad09f93002affe'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:06.799926 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1133/2282] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:16:06 2022] DELETE /identity/v3/users/82c31197ddc14ffe8bad09f93002affe => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:16:07.349601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cba8da-df96-4521-ab8c-aa0cf90c17c0 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:07.349601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-90cba8da-df96-4521-ab8c-aa0cf90c17c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:07.363253 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cba8da-df96-4521-ab8c-aa0cf90c17c0 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:07.448116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479c836-8141-4464-85ee-da8e606bb8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:07.449140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479c836-8141-4464-85ee-da8e606bb8b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:07.449426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479c836-8141-4464-85ee-da8e606bb8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:07.449702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479c836-8141-4464-85ee-da8e606bb8b0 admin admin] PATH_INFO: `/v3/projects/df5f885b55e84739bca9e718f2a95a84` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:07.450923 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479c836-8141-4464-85ee-da8e606bb8b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df5f885b55e84739bca9e718f2a95a84)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:07.453087 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7479c836-8141-4464-85ee-da8e606bb8b0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:07.526538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7479c836-8141-4464-85ee-da8e606bb8b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df5f885b55e84739bca9e718f2a95a84', 'request_id': 'req-7479c836-8141-4464-85ee-da8e606bb8b0'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:07.538293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7479c836-8141-4464-85ee-da8e606bb8b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df5f885b55e84739bca9e718f2a95a84', 'request_id': 'req-7479c836-8141-4464-85ee-da8e606bb8b0'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:07.549389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7479c836-8141-4464-85ee-da8e606bb8b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df5f885b55e84739bca9e718f2a95a84', 'request_id': 'req-7479c836-8141-4464-85ee-da8e606bb8b0'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:07.576516 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7479c836-8141-4464-85ee-da8e606bb8b0 admin admin] The token cache is being invalidate because project df5f885b55e84739bca9e718f2a95a84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:16:07.577176 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7479c836-8141-4464-85ee-da8e606bb8b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:07.578392 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7479c836-8141-4464-85ee-da8e606bb8b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:07.586482 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1150/2283] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:16:07 2022] DELETE /identity/v3/projects/df5f885b55e84739bca9e718f2a95a84 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:16:07.587518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2782f-e8ef-4672-bf2d-c6a1e82ff678 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:07.589062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2bd2782f-e8ef-4672-bf2d-c6a1e82ff678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:07.613199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2782f-e8ef-4672-bf2d-c6a1e82ff678 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:07.717979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8969d4-26ff-490c-b315-237e82a741b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:07.719180 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8969d4-26ff-490c-b315-237e82a741b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:07.719342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8969d4-26ff-490c-b315-237e82a741b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:07.719635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8969d4-26ff-490c-b315-237e82a741b7 admin admin] PATH_INFO: `/v3/users/2c66259ee6604f689f489bd891d449a6` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:07.721057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8969d4-26ff-490c-b315-237e82a741b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c66259ee6604f689f489bd891d449a6)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:07.723124 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8969d4-26ff-490c-b315-237e82a741b7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:07.758121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:07.798535 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6a8969d4-26ff-490c-b315-237e82a741b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c66259ee6604f689f489bd891d449a6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:07.800160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6a8969d4-26ff-490c-b315-237e82a741b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c66259ee6604f689f489bd891d449a6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:07.824210 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6a8969d4-26ff-490c-b315-237e82a741b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c66259ee6604f689f489bd891d449a6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:07.844573 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6a8969d4-26ff-490c-b315-237e82a741b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c66259ee6604f689f489bd891d449a6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:07.860295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-6a8969d4-26ff-490c-b315-237e82a741b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c66259ee6604f689f489bd891d449a6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:07.870432 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1134/2284] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:16:07 2022] DELETE /identity/v3/users/2c66259ee6604f689f489bd891d449a6 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:16:08.400288 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479c836-8141-4464-85ee-da8e606bb8b0 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:08.403348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7479c836-8141-4464-85ee-da8e606bb8b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:08.423995 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479c836-8141-4464-85ee-da8e606bb8b0 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:08.454739 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:08.515786 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9de6d-7ceb-4946-919e-953f868b9904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:08.516703 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9de6d-7ceb-4946-919e-953f868b9904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:08.517088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9de6d-7ceb-4946-919e-953f868b9904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:08.517422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9de6d-7ceb-4946-919e-953f868b9904 admin admin] PATH_INFO: `/v3/projects/f76fb6e70226460590f79ad0746967ca` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:08.520361 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9de6d-7ceb-4946-919e-953f868b9904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f76fb6e70226460590f79ad0746967ca)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:08.527262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b9de6d-7ceb-4946-919e-953f868b9904 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:08.626397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-98b9de6d-7ceb-4946-919e-953f868b9904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f76fb6e70226460590f79ad0746967ca', 'request_id': 'req-98b9de6d-7ceb-4946-919e-953f868b9904'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:08.636207 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-98b9de6d-7ceb-4946-919e-953f868b9904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f76fb6e70226460590f79ad0746967ca', 'request_id': 'req-98b9de6d-7ceb-4946-919e-953f868b9904'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:08.642909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-98b9de6d-7ceb-4946-919e-953f868b9904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f76fb6e70226460590f79ad0746967ca', 'request_id': 'req-98b9de6d-7ceb-4946-919e-953f868b9904'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:08.660099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-98b9de6d-7ceb-4946-919e-953f868b9904 admin admin] The token cache is being invalidate because project f76fb6e70226460590f79ad0746967ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:16:08.660660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-98b9de6d-7ceb-4946-919e-953f868b9904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:08.661656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-98b9de6d-7ceb-4946-919e-953f868b9904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:08.665509 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1151/2285] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:16:08 2022] DELETE /identity/v3/projects/f76fb6e70226460590f79ad0746967ca => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:16:08.670649 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e712597-5bc5-42cf-8c41-d62091a93eff None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:08.671616 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e712597-5bc5-42cf-8c41-d62091a93eff None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:08.672519 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e712597-5bc5-42cf-8c41-d62091a93eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:08.695386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8e712597-5bc5-42cf-8c41-d62091a93eff None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:08.697229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8e712597-5bc5-42cf-8c41-d62091a93eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:08.772317 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1135/2286] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:08.780725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2aa39e-62fd-4ffd-9723-28adf4304d1d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:08.781131 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2aa39e-62fd-4ffd-9723-28adf4304d1d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:08.781479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2aa39e-62fd-4ffd-9723-28adf4304d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:08.801937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6a2aa39e-62fd-4ffd-9723-28adf4304d1d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:08.803324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6a2aa39e-62fd-4ffd-9723-28adf4304d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:08.816882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1152/2287] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:08.818361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e712597-5bc5-42cf-8c41-d62091a93eff None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:08.831226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e712597-5bc5-42cf-8c41-d62091a93eff None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:08.837218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43111a-dc51-4c7e-888b-ffc90b72c061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:08.837904 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43111a-dc51-4c7e-888b-ffc90b72c061 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:08.838482 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43111a-dc51-4c7e-888b-ffc90b72c061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:08.838482 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43111a-dc51-4c7e-888b-ffc90b72c061 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:08.839459 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43111a-dc51-4c7e-888b-ffc90b72c061 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:16:08.839770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43111a-dc51-4c7e-888b-ffc90b72c061 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:08.841768 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43111a-dc51-4c7e-888b-ffc90b72c061 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:08.857367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1136/2288] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:16:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:16:08.862365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2aa39e-62fd-4ffd-9723-28adf4304d1d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:08.875038 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2aa39e-62fd-4ffd-9723-28adf4304d1d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:08.880441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5781d7c-90ee-4248-a2ef-b983f4b7c0b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:08.881209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5781d7c-90ee-4248-a2ef-b983f4b7c0b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:08.881513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5781d7c-90ee-4248-a2ef-b983f4b7c0b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:08.881795 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5781d7c-90ee-4248-a2ef-b983f4b7c0b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:08.882509 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5781d7c-90ee-4248-a2ef-b983f4b7c0b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:08.884451 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5781d7c-90ee-4248-a2ef-b983f4b7c0b9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:08.922133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1153/2289] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:16:08 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:16:08.924419 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43111a-dc51-4c7e-888b-ffc90b72c061 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:08.936524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43111a-dc51-4c7e-888b-ffc90b72c061 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:09.014555 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaeacd5-56d1-4f80-8ad1-064daf6ce5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:09.015236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaeacd5-56d1-4f80-8ad1-064daf6ce5f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:09.015829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaeacd5-56d1-4f80-8ad1-064daf6ce5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:09.016114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaeacd5-56d1-4f80-8ad1-064daf6ce5f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:09.016831 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaeacd5-56d1-4f80-8ad1-064daf6ce5f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:09.019319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaeacd5-56d1-4f80-8ad1-064daf6ce5f1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:09.060121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1137/2290] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:16:08 2022] POST /identity/v3/users => generated 291 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:16:09.060917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5781d7c-90ee-4248-a2ef-b983f4b7c0b9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:09.072692 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5781d7c-90ee-4248-a2ef-b983f4b7c0b9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:09.082959 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542b7d9-5ef9-4776-9b57-f8fd81853d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:09.083650 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542b7d9-5ef9-4776-9b57-f8fd81853d43 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:09.083924 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542b7d9-5ef9-4776-9b57-f8fd81853d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:09.084279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4542b7d9-5ef9-4776-9b57-f8fd81853d43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:09.085503 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542b7d9-5ef9-4776-9b57-f8fd81853d43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:09.088663 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4542b7d9-5ef9-4776-9b57-f8fd81853d43 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:09.102794 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1154/2291] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:16:09.107709 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaeacd5-56d1-4f80-8ad1-064daf6ce5f1 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:09.119809 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaeacd5-56d1-4f80-8ad1-064daf6ce5f1 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:09.124682 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee8d22e-9965-4803-8bba-acc10ccc1e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:09.125545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8d22e-9965-4803-8bba-acc10ccc1e83 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:09.125896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8d22e-9965-4803-8bba-acc10ccc1e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:09.126244 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee8d22e-9965-4803-8bba-acc10ccc1e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:09.127099 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8d22e-9965-4803-8bba-acc10ccc1e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:09.128771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee8d22e-9965-4803-8bba-acc10ccc1e83 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:09.146736 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542b7d9-5ef9-4776-9b57-f8fd81853d43 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:09.146952 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1138/2292] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:16:09.158841 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4542b7d9-5ef9-4776-9b57-f8fd81853d43 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:09.176247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e26c2f-e1f5-45d0-ad23-97c325ca8871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:09.183699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e26c2f-e1f5-45d0-ad23-97c325ca8871 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:09.184101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e26c2f-e1f5-45d0-ad23-97c325ca8871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:09.184591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e26c2f-e1f5-45d0-ad23-97c325ca8871 admin admin] PATH_INFO: `/v3/projects/a2eeff0739c1463383b618c4c56ae344/users/9eebb3ed7b2342efbd3eb4e72329c1e3/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:09.202999 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e26c2f-e1f5-45d0-ad23-97c325ca8871 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2eeff0739c1463383b618c4c56ae344, user_id=9eebb3ed7b2342efbd3eb4e72329c1e3, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:09.207344 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e26c2f-e1f5-45d0-ad23-97c325ca8871 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:09.229420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1155/2293] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:16:09 2022] PUT /identity/v3/projects/a2eeff0739c1463383b618c4c56ae344/users/9eebb3ed7b2342efbd3eb4e72329c1e3/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:16:09.235245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2437d-42ff-4f07-b6ee-c42412d19728 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:09.235707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2437d-42ff-4f07-b6ee-c42412d19728 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:09.236105 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2437d-42ff-4f07-b6ee-c42412d19728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:09.245769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:09.268393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7cd2437d-42ff-4f07-b6ee-c42412d19728 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:09.269784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7cd2437d-42ff-4f07-b6ee-c42412d19728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:09.353254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1139/2294] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:09.370071 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e26c2f-e1f5-45d0-ad23-97c325ca8871 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:09.372549 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-91e26c2f-e1f5-45d0-ad23-97c325ca8871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:09.523846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:09.532250 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e26c2f-e1f5-45d0-ad23-97c325ca8871 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:09.545535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685893e-7c92-42c1-b0ea-1ab46d970566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:09.546788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685893e-7c92-42c1-b0ea-1ab46d970566 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:09.548103 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685893e-7c92-42c1-b0ea-1ab46d970566 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:09.548374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9685893e-7c92-42c1-b0ea-1ab46d970566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:09.583141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685893e-7c92-42c1-b0ea-1ab46d970566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:09.586803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9685893e-7c92-42c1-b0ea-1ab46d970566 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:09.607703 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1156/2295] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:16:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:16:10.652857 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bbbff-9417-4c2c-9d43-07cb847bbc4a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:10.653575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bbbff-9417-4c2c-9d43-07cb847bbc4a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:10.653575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bbbff-9417-4c2c-9d43-07cb847bbc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:10.700884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-002bbbff-9417-4c2c-9d43-07cb847bbc4a None None] MFA Rules not processed for user `9eebb3ed7b2342efbd3eb4e72329c1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:10.702198 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-002bbbff-9417-4c2c-9d43-07cb847bbc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:10.860869 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1140/2296] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:10.872815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685893e-7c92-42c1-b0ea-1ab46d970566 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:10.874308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9685893e-7c92-42c1-b0ea-1ab46d970566 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:11.004877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9685893e-7c92-42c1-b0ea-1ab46d970566 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:11.013333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eee7040-1b7a-4cd5-87da-6b8aca267082 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:11.014017 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee7040-1b7a-4cd5-87da-6b8aca267082 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:11.014395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee7040-1b7a-4cd5-87da-6b8aca267082 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:11.014712 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eee7040-1b7a-4cd5-87da-6b8aca267082 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:11.034934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee7040-1b7a-4cd5-87da-6b8aca267082 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:11.035704 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eee7040-1b7a-4cd5-87da-6b8aca267082 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:11.062882 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035a63d-62a7-48b2-85e7-2c7f25989bcc None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:11.063027 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1157/2297] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:16:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:16:11.063121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035a63d-62a7-48b2-85e7-2c7f25989bcc None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:11.063217 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035a63d-62a7-48b2-85e7-2c7f25989bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:11.110094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b035a63d-62a7-48b2-85e7-2c7f25989bcc None None] MFA Rules not processed for user `ca339e26746a4feaaa25b14be59479a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:11.113226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b035a63d-62a7-48b2-85e7-2c7f25989bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:11.250893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1141/2298] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:13.699879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b9287-3261-43b6-8a01-93ec9c9287ee None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:13.701349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b9287-3261-43b6-8a01-93ec9c9287ee None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:13.702098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b9287-3261-43b6-8a01-93ec9c9287ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:13.729079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-132b9287-3261-43b6-8a01-93ec9c9287ee None None] MFA Rules not processed for user `9eebb3ed7b2342efbd3eb4e72329c1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:13.730502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-132b9287-3261-43b6-8a01-93ec9c9287ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:13.754966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1158/2299] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:21.215864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=82 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:23.719224 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=33 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:23.732411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736080f8-95d4-4fe3-8182-ece914138690 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:23.732886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736080f8-95d4-4fe3-8182-ece914138690 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:23.733280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736080f8-95d4-4fe3-8182-ece914138690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:23.751830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-736080f8-95d4-4fe3-8182-ece914138690 None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:23.753076 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-736080f8-95d4-4fe3-8182-ece914138690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:23.810309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1142/2300] 10.222.0.61 () {58 vars in 967 bytes} [Thu Jul 14 10:16:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:23.823130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b9287-3261-43b6-8a01-93ec9c9287ee None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:23.824513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-132b9287-3261-43b6-8a01-93ec9c9287ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:23.897580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b9287-3261-43b6-8a01-93ec9c9287ee None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:23.902939 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ef2d1-a061-443f-a1e9-5c53e5f96a71 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:23.903597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ef2d1-a061-443f-a1e9-5c53e5f96a71 service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:23.903880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ef2d1-a061-443f-a1e9-5c53e5f96a71 service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:23.904156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57ef2d1-a061-443f-a1e9-5c53e5f96a71 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:23.913750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ef2d1-a061-443f-a1e9-5c53e5f96a71 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:23.915658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57ef2d1-a061-443f-a1e9-5c53e5f96a71 service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:23.925237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1159/2301] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:16:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:16:26.177239 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736080f8-95d4-4fe3-8182-ece914138690 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:26.179487 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-736080f8-95d4-4fe3-8182-ece914138690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:26.217426 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736080f8-95d4-4fe3-8182-ece914138690 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:26.224523 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace71999-3ad0-42ac-8ced-81fe19fc050a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:26.226181 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace71999-3ad0-42ac-8ced-81fe19fc050a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:26.226561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace71999-3ad0-42ac-8ced-81fe19fc050a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:26.227110 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace71999-3ad0-42ac-8ced-81fe19fc050a admin admin] PATH_INFO: `/v3/users/5298be027cf843638957c4a15003db7f` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:26.229405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace71999-3ad0-42ac-8ced-81fe19fc050a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5298be027cf843638957c4a15003db7f)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:26.231486 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace71999-3ad0-42ac-8ced-81fe19fc050a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:26.291738 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ace71999-3ad0-42ac-8ced-81fe19fc050a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5298be027cf843638957c4a15003db7f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:26.292499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ace71999-3ad0-42ac-8ced-81fe19fc050a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5298be027cf843638957c4a15003db7f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:26.306057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ace71999-3ad0-42ac-8ced-81fe19fc050a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5298be027cf843638957c4a15003db7f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:26.313194 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ace71999-3ad0-42ac-8ced-81fe19fc050a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5298be027cf843638957c4a15003db7f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:26.322061 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ace71999-3ad0-42ac-8ced-81fe19fc050a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5298be027cf843638957c4a15003db7f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:26.328509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1143/2302] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:16:26 2022] DELETE /identity/v3/users/5298be027cf843638957c4a15003db7f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:16:26.509966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ef2d1-a061-443f-a1e9-5c53e5f96a71 service swift] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:26.523129 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57ef2d1-a061-443f-a1e9-5c53e5f96a71 service swift] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:26.585572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f2879d-3554-44f0-8e5e-4e42a6ae154c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:26.586439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2879d-3554-44f0-8e5e-4e42a6ae154c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:26.586716 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2879d-3554-44f0-8e5e-4e42a6ae154c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:26.586985 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f2879d-3554-44f0-8e5e-4e42a6ae154c admin admin] PATH_INFO: `/v3/projects/f523b9be668a4dc69619027cec280a08` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:26.588170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2879d-3554-44f0-8e5e-4e42a6ae154c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f523b9be668a4dc69619027cec280a08)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:26.590118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f2879d-3554-44f0-8e5e-4e42a6ae154c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:26.636954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-54f2879d-3554-44f0-8e5e-4e42a6ae154c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f523b9be668a4dc69619027cec280a08', 'request_id': 'req-54f2879d-3554-44f0-8e5e-4e42a6ae154c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:26.643908 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-54f2879d-3554-44f0-8e5e-4e42a6ae154c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f523b9be668a4dc69619027cec280a08', 'request_id': 'req-54f2879d-3554-44f0-8e5e-4e42a6ae154c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:26.650098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-54f2879d-3554-44f0-8e5e-4e42a6ae154c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f523b9be668a4dc69619027cec280a08', 'request_id': 'req-54f2879d-3554-44f0-8e5e-4e42a6ae154c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:26.663489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-54f2879d-3554-44f0-8e5e-4e42a6ae154c admin admin] The token cache is being invalidate because project f523b9be668a4dc69619027cec280a08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:16:26.663775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-54f2879d-3554-44f0-8e5e-4e42a6ae154c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:26.664410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-54f2879d-3554-44f0-8e5e-4e42a6ae154c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:26.666518 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1160/2303] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:16:26 2022] DELETE /identity/v3/projects/f523b9be668a4dc69619027cec280a08 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:16:26.671558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec450b-8335-4edf-940e-d1fd31bc56b8 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:26.671852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec450b-8335-4edf-940e-d1fd31bc56b8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:26.672181 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec450b-8335-4edf-940e-d1fd31bc56b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:26.690863 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-27ec450b-8335-4edf-940e-d1fd31bc56b8 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:26.692052 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-27ec450b-8335-4edf-940e-d1fd31bc56b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:26.756746 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1144/2304] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:26.764128 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10b668-597e-4e15-b653-2945290b9abc None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:26.764420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10b668-597e-4e15-b653-2945290b9abc None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:26.764699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10b668-597e-4e15-b653-2945290b9abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:26.786295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0d10b668-597e-4e15-b653-2945290b9abc None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:26.788867 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0d10b668-597e-4e15-b653-2945290b9abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:26.797623 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec450b-8335-4edf-940e-d1fd31bc56b8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:26.799506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-27ec450b-8335-4edf-940e-d1fd31bc56b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:26.804088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1161/2305] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:26.805525 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10b668-597e-4e15-b653-2945290b9abc None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:26.814274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec450b-8335-4edf-940e-d1fd31bc56b8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:26.816126 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10b668-597e-4e15-b653-2945290b9abc None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:26.819156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf03267-a243-4090-8c7b-00632fc165bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:26.820142 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf03267-a243-4090-8c7b-00632fc165bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:26.820491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf03267-a243-4090-8c7b-00632fc165bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:26.820828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf03267-a243-4090-8c7b-00632fc165bb admin admin] PATH_INFO: `/v3/users/938620e3c8064c59b5c316bfcf535cbd` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:26.820962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e844bc6-e7c6-4c6d-98da-0e55f0de89ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:26.821571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e844bc6-e7c6-4c6d-98da-0e55f0de89ad admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:26.821854 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e844bc6-e7c6-4c6d-98da-0e55f0de89ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:26.822129 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e844bc6-e7c6-4c6d-98da-0e55f0de89ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:26.822500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf03267-a243-4090-8c7b-00632fc165bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=938620e3c8064c59b5c316bfcf535cbd)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:26.822858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e844bc6-e7c6-4c6d-98da-0e55f0de89ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:16:26.823173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e844bc6-e7c6-4c6d-98da-0e55f0de89ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:26.824595 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf03267-a243-4090-8c7b-00632fc165bb admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:26.824760 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e844bc6-e7c6-4c6d-98da-0e55f0de89ad admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:26.835123 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:26.842340 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:26.842971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1162/2306] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:16:26.843852 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e844bc6-e7c6-4c6d-98da-0e55f0de89ad admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:26.856499 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e844bc6-e7c6-4c6d-98da-0e55f0de89ad admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:26.863122 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12db94-19eb-4b2d-ac27-92e14220f73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:26.863855 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12db94-19eb-4b2d-ac27-92e14220f73d admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:26.864211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12db94-19eb-4b2d-ac27-92e14220f73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:26.864559 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc12db94-19eb-4b2d-ac27-92e14220f73d admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:26.865433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12db94-19eb-4b2d-ac27-92e14220f73d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:26.867637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc12db94-19eb-4b2d-ac27-92e14220f73d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:26.888396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-0cf03267-a243-4090-8c7b-00632fc165bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '938620e3c8064c59b5c316bfcf535cbd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:26.889029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-0cf03267-a243-4090-8c7b-00632fc165bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '938620e3c8064c59b5c316bfcf535cbd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:26.899048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1163/2307] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:16:26 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:16:26.903459 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-0cf03267-a243-4090-8c7b-00632fc165bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '938620e3c8064c59b5c316bfcf535cbd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:26.903902 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12db94-19eb-4b2d-ac27-92e14220f73d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:26.905889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bc12db94-19eb-4b2d-ac27-92e14220f73d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:26.910685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-0cf03267-a243-4090-8c7b-00632fc165bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '938620e3c8064c59b5c316bfcf535cbd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:26.920801 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-0cf03267-a243-4090-8c7b-00632fc165bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '938620e3c8064c59b5c316bfcf535cbd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:26.927280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1145/2308] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:16:26 2022] DELETE /identity/v3/users/938620e3c8064c59b5c316bfcf535cbd => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:16:26.931466 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f38c2-fff2-4b58-8572-52a10f0a1ebe None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:26.931756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f38c2-fff2-4b58-8572-52a10f0a1ebe None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:26.932033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9f38c2-fff2-4b58-8572-52a10f0a1ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:26.953469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3f9f38c2-fff2-4b58-8572-52a10f0a1ebe None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:26.954692 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3f9f38c2-fff2-4b58-8572-52a10f0a1ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:26.992064 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc12db94-19eb-4b2d-ac27-92e14220f73d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:27.035373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1146/2309] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:16:27.046741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f38c2-fff2-4b58-8572-52a10f0a1ebe None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:27.050636 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3f9f38c2-fff2-4b58-8572-52a10f0a1ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:27.104390 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789abe4-5992-42a0-94a7-3c8274852b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:27.105929 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789abe4-5992-42a0-94a7-3c8274852b59 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:27.106401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789abe4-5992-42a0-94a7-3c8274852b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:27.106825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789abe4-5992-42a0-94a7-3c8274852b59 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:27.108147 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789abe4-5992-42a0-94a7-3c8274852b59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:27.111502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789abe4-5992-42a0-94a7-3c8274852b59 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:27.144089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9f38c2-fff2-4b58-8572-52a10f0a1ebe None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:27.149919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ce50e-863c-4ad5-9e38-8281ea6d5edf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:27.151208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ce50e-863c-4ad5-9e38-8281ea6d5edf service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:27.151760 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ce50e-863c-4ad5-9e38-8281ea6d5edf service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:27.152039 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64ce50e-863c-4ad5-9e38-8281ea6d5edf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:27.161927 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1164/2310] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:16:26 2022] POST /identity/v3/users => generated 287 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:16:27.164213 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ce50e-863c-4ad5-9e38-8281ea6d5edf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:27.165738 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64ce50e-863c-4ad5-9e38-8281ea6d5edf service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:27.166183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789abe4-5992-42a0-94a7-3c8274852b59 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:27.176510 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1147/2311] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:16:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:16:27.187786 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789abe4-5992-42a0-94a7-3c8274852b59 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:27.195052 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977091a-17f2-48c1-8319-6cafbafb6626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:27.195832 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977091a-17f2-48c1-8319-6cafbafb6626 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:27.196200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977091a-17f2-48c1-8319-6cafbafb6626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:27.196547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b977091a-17f2-48c1-8319-6cafbafb6626 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:27.197512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977091a-17f2-48c1-8319-6cafbafb6626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:27.200772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b977091a-17f2-48c1-8319-6cafbafb6626 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:27.212991 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1165/2312] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:16:27.216299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ce50e-863c-4ad5-9e38-8281ea6d5edf service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:27.228344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64ce50e-863c-4ad5-9e38-8281ea6d5edf service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:27.238808 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e7da9-84d4-466e-99de-7ba578dbb5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:27.238808 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e7da9-84d4-466e-99de-7ba578dbb5bf admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:27.238808 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e7da9-84d4-466e-99de-7ba578dbb5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:27.238808 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e7da9-84d4-466e-99de-7ba578dbb5bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:27.239268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e7da9-84d4-466e-99de-7ba578dbb5bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:27.239268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e7da9-84d4-466e-99de-7ba578dbb5bf admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:27.250744 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1148/2313] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:16:27.254465 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977091a-17f2-48c1-8319-6cafbafb6626 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:27.265591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b977091a-17f2-48c1-8319-6cafbafb6626 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:27.271324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a833f-dde2-41df-bf4a-d35d41129b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:27.272337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a833f-dde2-41df-bf4a-d35d41129b80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:27.272619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a833f-dde2-41df-bf4a-d35d41129b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:27.272892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a833f-dde2-41df-bf4a-d35d41129b80 admin admin] PATH_INFO: `/v3/projects/47163a6d0e704f8eb8d0725284918f86/users/994f748d68504033a95385b1ab507d85/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:27.292664 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a833f-dde2-41df-bf4a-d35d41129b80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47163a6d0e704f8eb8d0725284918f86, user_id=994f748d68504033a95385b1ab507d85, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:27.295513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a833f-dde2-41df-bf4a-d35d41129b80 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:27.309756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1166/2314] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:16:27 2022] PUT /identity/v3/projects/47163a6d0e704f8eb8d0725284918f86/users/994f748d68504033a95385b1ab507d85/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:16:27.320083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6d214-9736-4421-9763-76fb36a9140c None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:27.320367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6d214-9736-4421-9763-76fb36a9140c None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:27.320640 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6d214-9736-4421-9763-76fb36a9140c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:27.347271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-40f6d214-9736-4421-9763-76fb36a9140c None None] MFA Rules not processed for user `994f748d68504033a95385b1ab507d85`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:27.348467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-40f6d214-9736-4421-9763-76fb36a9140c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:27.358636 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:27.420129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1149/2315] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:27.433101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a833f-dde2-41df-bf4a-d35d41129b80 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:27.434794 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d01a833f-dde2-41df-bf4a-d35d41129b80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:27.489005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:27.505032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a833f-dde2-41df-bf4a-d35d41129b80 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:27.509390 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55167bf9-4777-4cc7-b12a-153bce1f740a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:27.510041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55167bf9-4777-4cc7-b12a-153bce1f740a service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:27.510602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55167bf9-4777-4cc7-b12a-153bce1f740a service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:27.510602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55167bf9-4777-4cc7-b12a-153bce1f740a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:27.519510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55167bf9-4777-4cc7-b12a-153bce1f740a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:27.520951 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55167bf9-4777-4cc7-b12a-153bce1f740a service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:27.533481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1167/2316] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:16:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:16:27.851324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6d214-9736-4421-9763-76fb36a9140c None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:27.853401 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-40f6d214-9736-4421-9763-76fb36a9140c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:27.965457 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6d214-9736-4421-9763-76fb36a9140c None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:27.972896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4cb23-ccb3-49b0-9ad9-44c5a7d1fb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:27.974108 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4cb23-ccb3-49b0-9ad9-44c5a7d1fb76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:27.974576 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4cb23-ccb3-49b0-9ad9-44c5a7d1fb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:27.975608 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4cb23-ccb3-49b0-9ad9-44c5a7d1fb76 admin admin] PATH_INFO: `/v3/projects/4c6e169f09a3410084ca68a4a9fc9956` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:27.977046 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4cb23-ccb3-49b0-9ad9-44c5a7d1fb76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c6e169f09a3410084ca68a4a9fc9956)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:27.979435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4cb23-ccb3-49b0-9ad9-44c5a7d1fb76 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:28.048265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9df4cb23-ccb3-49b0-9ad9-44c5a7d1fb76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c6e169f09a3410084ca68a4a9fc9956', 'request_id': 'req-9df4cb23-ccb3-49b0-9ad9-44c5a7d1fb76'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:28.055654 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9df4cb23-ccb3-49b0-9ad9-44c5a7d1fb76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c6e169f09a3410084ca68a4a9fc9956', 'request_id': 'req-9df4cb23-ccb3-49b0-9ad9-44c5a7d1fb76'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:28.078563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9df4cb23-ccb3-49b0-9ad9-44c5a7d1fb76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c6e169f09a3410084ca68a4a9fc9956', 'request_id': 'req-9df4cb23-ccb3-49b0-9ad9-44c5a7d1fb76'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:28.097735 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9df4cb23-ccb3-49b0-9ad9-44c5a7d1fb76 admin admin] The token cache is being invalidate because project 4c6e169f09a3410084ca68a4a9fc9956 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:16:28.098169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9df4cb23-ccb3-49b0-9ad9-44c5a7d1fb76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:28.098927 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9df4cb23-ccb3-49b0-9ad9-44c5a7d1fb76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:16:28.102905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1150/2317] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:16:27 2022] DELETE /identity/v3/projects/4c6e169f09a3410084ca68a4a9fc9956 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:16:28.107783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dcdf8-54d7-44e8-95ca-54cdc5f5f352 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:28.108205 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dcdf8-54d7-44e8-95ca-54cdc5f5f352 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:28.108601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7dcdf8-54d7-44e8-95ca-54cdc5f5f352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:28.130269 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-aa7dcdf8-54d7-44e8-95ca-54cdc5f5f352 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:28.132407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-aa7dcdf8-54d7-44e8-95ca-54cdc5f5f352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:28.207667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1168/2318] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:16:28.215939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ab858-7e22-420f-a153-df025663dd3a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:28.216235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ab858-7e22-420f-a153-df025663dd3a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:28.216520 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ab858-7e22-420f-a153-df025663dd3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:28.253694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3b7ab858-7e22-420f-a153-df025663dd3a None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:28.255134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3b7ab858-7e22-420f-a153-df025663dd3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:28.267374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1151/2319] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:28.268474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dcdf8-54d7-44e8-95ca-54cdc5f5f352 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:28.280636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7dcdf8-54d7-44e8-95ca-54cdc5f5f352 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:28.285558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d05b66-3767-45e2-bc10-61d729dd2b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:28.286327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d05b66-3767-45e2-bc10-61d729dd2b69 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:28.286709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d05b66-3767-45e2-bc10-61d729dd2b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:28.287083 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d05b66-3767-45e2-bc10-61d729dd2b69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:28.287946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d05b66-3767-45e2-bc10-61d729dd2b69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:16:28.288379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d05b66-3767-45e2-bc10-61d729dd2b69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:28.290108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d05b66-3767-45e2-bc10-61d729dd2b69 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:28.305733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1169/2320] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:16:28.309369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ab858-7e22-420f-a153-df025663dd3a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:28.320097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ab858-7e22-420f-a153-df025663dd3a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:28.325522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7712d2-1962-4327-8200-c36440cd63bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:28.326875 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7712d2-1962-4327-8200-c36440cd63bc admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:28.327227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7712d2-1962-4327-8200-c36440cd63bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:28.327574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7712d2-1962-4327-8200-c36440cd63bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:28.328413 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7712d2-1962-4327-8200-c36440cd63bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:28.330941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7712d2-1962-4327-8200-c36440cd63bc admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:28.362450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1152/2321] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:16:28 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:16:28.363593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d05b66-3767-45e2-bc10-61d729dd2b69 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:28.374844 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d05b66-3767-45e2-bc10-61d729dd2b69 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:28.434249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159ada3-368f-40ab-aa2c-635d8243241f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:28.434949 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159ada3-368f-40ab-aa2c-635d8243241f admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:28.435232 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159ada3-368f-40ab-aa2c-635d8243241f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:28.435505 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159ada3-368f-40ab-aa2c-635d8243241f admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:28.436207 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159ada3-368f-40ab-aa2c-635d8243241f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:28.438301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159ada3-368f-40ab-aa2c-635d8243241f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:28.459942 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1170/2322] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:16:28 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:16:28.463988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7712d2-1962-4327-8200-c36440cd63bc admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:28.474381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7712d2-1962-4327-8200-c36440cd63bc admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:28.479124 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe6725-a0e8-4fe7-afc2-74404d1dbcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:28.479786 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe6725-a0e8-4fe7-afc2-74404d1dbcde admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:28.480070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe6725-a0e8-4fe7-afc2-74404d1dbcde admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:28.480343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe6725-a0e8-4fe7-afc2-74404d1dbcde admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:28.481113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe6725-a0e8-4fe7-afc2-74404d1dbcde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:28.482705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fe6725-a0e8-4fe7-afc2-74404d1dbcde admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:28.493978 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1153/2323] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:16:28.497863 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159ada3-368f-40ab-aa2c-635d8243241f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:28.508441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159ada3-368f-40ab-aa2c-635d8243241f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:28.514254 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9ce35-da1e-4cb6-9220-65bdd3aeaaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:28.514922 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9ce35-da1e-4cb6-9220-65bdd3aeaaee admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:28.515206 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9ce35-da1e-4cb6-9220-65bdd3aeaaee admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:28.515482 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9ce35-da1e-4cb6-9220-65bdd3aeaaee admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:28.516257 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9ce35-da1e-4cb6-9220-65bdd3aeaaee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:28.517919 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b9ce35-da1e-4cb6-9220-65bdd3aeaaee admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:28.528722 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1171/2324] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:16:28.532636 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe6725-a0e8-4fe7-afc2-74404d1dbcde admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:28.537904 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:28.543841 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe6725-a0e8-4fe7-afc2-74404d1dbcde admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:28.551543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316a8b8-63c5-47f8-973d-8e38a2588852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:28.552659 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316a8b8-63c5-47f8-973d-8e38a2588852 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:28.555626 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316a8b8-63c5-47f8-973d-8e38a2588852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:28.555911 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e316a8b8-63c5-47f8-973d-8e38a2588852 admin admin] PATH_INFO: `/v3/projects/524df7e140f74c69875de81de9ed7e32/users/b612d0e2c43444b991ea53cc996591eb/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:28.578518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316a8b8-63c5-47f8-973d-8e38a2588852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=524df7e140f74c69875de81de9ed7e32, user_id=b612d0e2c43444b991ea53cc996591eb, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:28.581199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e316a8b8-63c5-47f8-973d-8e38a2588852 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:28.601128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1154/2325] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:16:28 2022] PUT /identity/v3/projects/524df7e140f74c69875de81de9ed7e32/users/b612d0e2c43444b991ea53cc996591eb/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:16:28.607804 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd08480-f824-45b9-878a-3aee2e6576eb None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:28.608097 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd08480-f824-45b9-878a-3aee2e6576eb None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:28.608429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd08480-f824-45b9-878a-3aee2e6576eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:28.639965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ecd08480-f824-45b9-878a-3aee2e6576eb None None] MFA Rules not processed for user `b612d0e2c43444b991ea53cc996591eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:28.641267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ecd08480-f824-45b9-878a-3aee2e6576eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:28.691315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:28.722847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1172/2326] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:28.735096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316a8b8-63c5-47f8-973d-8e38a2588852 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:28.737062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e316a8b8-63c5-47f8-973d-8e38a2588852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:28.835763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e316a8b8-63c5-47f8-973d-8e38a2588852 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:28.840513 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d640bb-a3cf-4720-9ba3-38bf3f5b18c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:28.841586 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d640bb-a3cf-4720-9ba3-38bf3f5b18c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:28.842030 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d640bb-a3cf-4720-9ba3-38bf3f5b18c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:28.842381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d640bb-a3cf-4720-9ba3-38bf3f5b18c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:28.854315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d640bb-a3cf-4720-9ba3-38bf3f5b18c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:28.856064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d640bb-a3cf-4720-9ba3-38bf3f5b18c1 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:28.867065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1155/2327] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:16:28 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:16:28.996612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb64b67-c432-49fe-80f4-9c0eddfac110 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:28.996899 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb64b67-c432-49fe-80f4-9c0eddfac110 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:28.997196 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb64b67-c432-49fe-80f4-9c0eddfac110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:29.014733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ccb64b67-c432-49fe-80f4-9c0eddfac110 None None] MFA Rules not processed for user `994f748d68504033a95385b1ab507d85`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:29.016547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ccb64b67-c432-49fe-80f4-9c0eddfac110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:29.109415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1173/2328] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:16:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:38.862852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=35,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:39.089494 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=35 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:16:52.771309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d640bb-a3cf-4720-9ba3-38bf3f5b18c1 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:52.772829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-64d640bb-a3cf-4720-9ba3-38bf3f5b18c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:52.845501 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d640bb-a3cf-4720-9ba3-38bf3f5b18c1 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:52.851574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d87cae-c70c-465c-90cc-b74899d75cd0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff4c42a4f4f549378e5bc1cd92f801c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:52.852330 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d87cae-c70c-465c-90cc-b74899d75cd0 service placement] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:52.852614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d87cae-c70c-465c-90cc-b74899d75cd0 service placement] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:52.853754 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d87cae-c70c-465c-90cc-b74899d75cd0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:52.856211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-54d87cae-c70c-465c-90cc-b74899d75cd0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:52.871532 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d87cae-c70c-465c-90cc-b74899d75cd0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:52.873293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d87cae-c70c-465c-90cc-b74899d75cd0 service placement] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:52.884121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1156/2329] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:16:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:16:58.545050 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04534e-57a4-4ae6-beab-7e3151369b20 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:58.545577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04534e-57a4-4ae6-beab-7e3151369b20 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:58.545577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04534e-57a4-4ae6-beab-7e3151369b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:58.562883 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0f04534e-57a4-4ae6-beab-7e3151369b20 None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:58.564063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0f04534e-57a4-4ae6-beab-7e3151369b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:58.618867 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1174/2330] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Jul 14 10:16:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:58.630696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d87cae-c70c-465c-90cc-b74899d75cd0 service placement] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:58.632072 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-54d87cae-c70c-465c-90cc-b74899d75cd0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:58.700852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d87cae-c70c-465c-90cc-b74899d75cd0 service placement] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:58.705890 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8d696-638a-44e0-a25c-0921454dfc80 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:58.706532 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8d696-638a-44e0-a25c-0921454dfc80 service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:58.706831 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8d696-638a-44e0-a25c-0921454dfc80 service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:58.707091 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca8d696-638a-44e0-a25c-0921454dfc80 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:58.715592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8d696-638a-44e0-a25c-0921454dfc80 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:58.717189 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca8d696-638a-44e0-a25c-0921454dfc80 service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:58.725092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1157/2331] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:16:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:16:59.212517 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bf11b-b1d6-4284-a40a-ec0b7a972b48 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:59.212810 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bf11b-b1d6-4284-a40a-ec0b7a972b48 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:59.213135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160bf11b-b1d6-4284-a40a-ec0b7a972b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:59.233595 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-160bf11b-b1d6-4284-a40a-ec0b7a972b48 None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:59.234824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-160bf11b-b1d6-4284-a40a-ec0b7a972b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:59.242079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1175/2332] 10.222.0.61 () {58 vars in 967 bytes} [Thu Jul 14 10:16:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:59.254817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8d696-638a-44e0-a25c-0921454dfc80 service swift] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:59.266104 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca8d696-638a-44e0-a25c-0921454dfc80 service swift] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:59.271141 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37045830-cd9e-4792-8fb0-e32a8c167c0f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:59.271720 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37045830-cd9e-4792-8fb0-e32a8c167c0f service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:59.272000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37045830-cd9e-4792-8fb0-e32a8c167c0f service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:59.272272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37045830-cd9e-4792-8fb0-e32a8c167c0f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:59.281050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37045830-cd9e-4792-8fb0-e32a8c167c0f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:59.282608 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37045830-cd9e-4792-8fb0-e32a8c167c0f service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:59.291043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1158/2333] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:16:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:16:59.482128 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480eb51c-5a26-4f05-9bc0-d93ae6c7323b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:59.482401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480eb51c-5a26-4f05-9bc0-d93ae6c7323b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:59.482677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480eb51c-5a26-4f05-9bc0-d93ae6c7323b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:59.500941 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-480eb51c-5a26-4f05-9bc0-d93ae6c7323b None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:59.502173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-480eb51c-5a26-4f05-9bc0-d93ae6c7323b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:59.509949 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1176/2334] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Jul 14 10:16:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:59.523966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37045830-cd9e-4792-8fb0-e32a8c167c0f service swift] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:59.535406 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37045830-cd9e-4792-8fb0-e32a8c167c0f service swift] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:59.540653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56add9f-2661-44f3-8a7c-a0ff131391e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:59.541340 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56add9f-2661-44f3-8a7c-a0ff131391e0 service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:59.541619 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56add9f-2661-44f3-8a7c-a0ff131391e0 service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:59.542768 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56add9f-2661-44f3-8a7c-a0ff131391e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:59.550294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56add9f-2661-44f3-8a7c-a0ff131391e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:59.551848 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56add9f-2661-44f3-8a7c-a0ff131391e0 service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:59.559757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1159/2335] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:16:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:16:59.863935 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3b831-38a2-40a4-8a2d-fe0fa0b4196b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:59.864388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3b831-38a2-40a4-8a2d-fe0fa0b4196b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:59.864502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd3b831-38a2-40a4-8a2d-fe0fa0b4196b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:59.881857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-fcd3b831-38a2-40a4-8a2d-fe0fa0b4196b None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:16:59.883049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fcd3b831-38a2-40a4-8a2d-fe0fa0b4196b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:16:59.890102 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1177/2336] 10.222.0.61 () {58 vars in 967 bytes} [Thu Jul 14 10:16:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:16:59.902936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56add9f-2661-44f3-8a7c-a0ff131391e0 service swift] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:16:59.913328 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56add9f-2661-44f3-8a7c-a0ff131391e0 service swift] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:16:59.919031 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb8bf6-13af-4a2d-9fbf-f6db81eb03f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:16:59.919667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb8bf6-13af-4a2d-9fbf-f6db81eb03f1 service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:16:59.919948 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb8bf6-13af-4a2d-9fbf-f6db81eb03f1 service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:16:59.920224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb8bf6-13af-4a2d-9fbf-f6db81eb03f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:16:59.928292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb8bf6-13af-4a2d-9fbf-f6db81eb03f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:16:59.929938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb8bf6-13af-4a2d-9fbf-f6db81eb03f1 service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:16:59.938814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1160/2337] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:16:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:17:01.995955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3b831-38a2-40a4-8a2d-fe0fa0b4196b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:01.997472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fcd3b831-38a2-40a4-8a2d-fe0fa0b4196b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:02.064045 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd3b831-38a2-40a4-8a2d-fe0fa0b4196b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:02.068448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b299ffc-c80f-4ed1-93a8-2bb5f5f4dca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:02.069399 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b299ffc-c80f-4ed1-93a8-2bb5f5f4dca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:02.069677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b299ffc-c80f-4ed1-93a8-2bb5f5f4dca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:02.069946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b299ffc-c80f-4ed1-93a8-2bb5f5f4dca0 admin admin] PATH_INFO: `/v3/users/994f748d68504033a95385b1ab507d85` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:02.071156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b299ffc-c80f-4ed1-93a8-2bb5f5f4dca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=994f748d68504033a95385b1ab507d85)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:02.073122 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b299ffc-c80f-4ed1-93a8-2bb5f5f4dca0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:02.122037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4b299ffc-c80f-4ed1-93a8-2bb5f5f4dca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '994f748d68504033a95385b1ab507d85'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:02.122713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4b299ffc-c80f-4ed1-93a8-2bb5f5f4dca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '994f748d68504033a95385b1ab507d85'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:02.134826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4b299ffc-c80f-4ed1-93a8-2bb5f5f4dca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '994f748d68504033a95385b1ab507d85'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:02.141416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4b299ffc-c80f-4ed1-93a8-2bb5f5f4dca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '994f748d68504033a95385b1ab507d85'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:02.148685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4b299ffc-c80f-4ed1-93a8-2bb5f5f4dca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '994f748d68504033a95385b1ab507d85'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:02.162231 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1178/2338] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:17:01 2022] DELETE /identity/v3/users/994f748d68504033a95385b1ab507d85 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:17:02.167425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566893f5-8204-4d88-b9c8-c4195e6e458f None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:02.167877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566893f5-8204-4d88-b9c8-c4195e6e458f None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:02.168134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566893f5-8204-4d88-b9c8-c4195e6e458f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:02.186873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-566893f5-8204-4d88-b9c8-c4195e6e458f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:17:02.188146 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-566893f5-8204-4d88-b9c8-c4195e6e458f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:02.222845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:17:02.248862 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1161/2339] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:17:02.258282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b299ffc-c80f-4ed1-93a8-2bb5f5f4dca0 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:02.260359 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4b299ffc-c80f-4ed1-93a8-2bb5f5f4dca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:02.267004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:17:02.320504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b299ffc-c80f-4ed1-93a8-2bb5f5f4dca0 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:02.324777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ff8116-b012-497e-ac86-28d95b995e3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:02.325458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff8116-b012-497e-ac86-28d95b995e3a service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:02.325735 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff8116-b012-497e-ac86-28d95b995e3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:02.326006 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ff8116-b012-497e-ac86-28d95b995e3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:02.334871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff8116-b012-497e-ac86-28d95b995e3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:02.336342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ff8116-b012-497e-ac86-28d95b995e3a service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:02.348093 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1179/2340] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:17:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:17:02.522728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566893f5-8204-4d88-b9c8-c4195e6e458f None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:02.524178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-566893f5-8204-4d88-b9c8-c4195e6e458f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:02.536625 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566893f5-8204-4d88-b9c8-c4195e6e458f None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:02.541817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3bb5e8-a48c-48a1-8500-3d3c4f7081f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:02.542676 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bb5e8-a48c-48a1-8500-3d3c4f7081f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:02.542953 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bb5e8-a48c-48a1-8500-3d3c4f7081f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:02.543227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3bb5e8-a48c-48a1-8500-3d3c4f7081f6 admin admin] PATH_INFO: `/v3/projects/47163a6d0e704f8eb8d0725284918f86` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:02.544418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3bb5e8-a48c-48a1-8500-3d3c4f7081f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47163a6d0e704f8eb8d0725284918f86)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:02.546407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3bb5e8-a48c-48a1-8500-3d3c4f7081f6 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:02.598286 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2c3bb5e8-a48c-48a1-8500-3d3c4f7081f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47163a6d0e704f8eb8d0725284918f86', 'request_id': 'req-2c3bb5e8-a48c-48a1-8500-3d3c4f7081f6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:02.606588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2c3bb5e8-a48c-48a1-8500-3d3c4f7081f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47163a6d0e704f8eb8d0725284918f86', 'request_id': 'req-2c3bb5e8-a48c-48a1-8500-3d3c4f7081f6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:02.617319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2c3bb5e8-a48c-48a1-8500-3d3c4f7081f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47163a6d0e704f8eb8d0725284918f86', 'request_id': 'req-2c3bb5e8-a48c-48a1-8500-3d3c4f7081f6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:02.634455 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2c3bb5e8-a48c-48a1-8500-3d3c4f7081f6 admin admin] The token cache is being invalidate because project 47163a6d0e704f8eb8d0725284918f86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:17:02.635006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2c3bb5e8-a48c-48a1-8500-3d3c4f7081f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:02.636060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2c3bb5e8-a48c-48a1-8500-3d3c4f7081f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:02.637763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1162/2341] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:17:02 2022] DELETE /identity/v3/projects/47163a6d0e704f8eb8d0725284918f86 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:17:02.643185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026cc2e-16ea-460b-b12b-7c6a4ea34a00 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:02.643470 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026cc2e-16ea-460b-b12b-7c6a4ea34a00 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:02.643742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7026cc2e-16ea-460b-b12b-7c6a4ea34a00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:02.662599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7026cc2e-16ea-460b-b12b-7c6a4ea34a00 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:17:02.663858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7026cc2e-16ea-460b-b12b-7c6a4ea34a00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:02.726916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1180/2342] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:17:02.736032 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fd48e-dbef-43c5-bb3f-38f210355e65 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:02.736325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fd48e-dbef-43c5-bb3f-38f210355e65 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:02.736610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fd48e-dbef-43c5-bb3f-38f210355e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:02.755252 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-008fd48e-dbef-43c5-bb3f-38f210355e65 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:17:02.756453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-008fd48e-dbef-43c5-bb3f-38f210355e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:02.764332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1163/2343] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:17:02.768379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026cc2e-16ea-460b-b12b-7c6a4ea34a00 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:02.778707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7026cc2e-16ea-460b-b12b-7c6a4ea34a00 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:02.783313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac65a23-950e-44a9-897d-b466af9e95ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:02.783962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac65a23-950e-44a9-897d-b466af9e95ef admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:02.784246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac65a23-950e-44a9-897d-b466af9e95ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:02.784559 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac65a23-950e-44a9-897d-b466af9e95ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:02.785327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac65a23-950e-44a9-897d-b466af9e95ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:17:02.785640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac65a23-950e-44a9-897d-b466af9e95ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:02.787228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac65a23-950e-44a9-897d-b466af9e95ef admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:02.799332 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1181/2344] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:17:02.803402 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fd48e-dbef-43c5-bb3f-38f210355e65 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:02.812602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fd48e-dbef-43c5-bb3f-38f210355e65 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:02.817293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a161e-8ca7-4ca4-bc34-57465f178bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:02.818080 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a161e-8ca7-4ca4-bc34-57465f178bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:02.818360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a161e-8ca7-4ca4-bc34-57465f178bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:02.818767 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a161e-8ca7-4ca4-bc34-57465f178bd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:02.819591 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a161e-8ca7-4ca4-bc34-57465f178bd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:02.821872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86a161e-8ca7-4ca4-bc34-57465f178bd2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:02.849241 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1164/2345] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:17:02 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:17:02.853840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac65a23-950e-44a9-897d-b466af9e95ef admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:02.863658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac65a23-950e-44a9-897d-b466af9e95ef admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:02.921707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2474a5-7649-48ca-b92a-4e26a33fa2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:02.922384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2474a5-7649-48ca-b92a-4e26a33fa2b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:02.922672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2474a5-7649-48ca-b92a-4e26a33fa2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:02.922945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2474a5-7649-48ca-b92a-4e26a33fa2b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:02.923635 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2474a5-7649-48ca-b92a-4e26a33fa2b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:02.925588 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2474a5-7649-48ca-b92a-4e26a33fa2b9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:02.944964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1182/2346] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:17:02 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:17:02.949198 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a161e-8ca7-4ca4-bc34-57465f178bd2 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:02.959313 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86a161e-8ca7-4ca4-bc34-57465f178bd2 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:02.964380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a152bd-4360-4f71-b41d-17130421bbe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:02.965064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a152bd-4360-4f71-b41d-17130421bbe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:02.965344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a152bd-4360-4f71-b41d-17130421bbe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:02.965611 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a152bd-4360-4f71-b41d-17130421bbe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:02.966330 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a152bd-4360-4f71-b41d-17130421bbe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:02.968121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a152bd-4360-4f71-b41d-17130421bbe6 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:02.980369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1165/2347] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:17:02.984598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2474a5-7649-48ca-b92a-4e26a33fa2b9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:02.995226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2474a5-7649-48ca-b92a-4e26a33fa2b9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:02.999917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e087ee47-1768-404b-81f8-7a6de25ccca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:03.000578 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087ee47-1768-404b-81f8-7a6de25ccca6 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:03.000865 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087ee47-1768-404b-81f8-7a6de25ccca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:03.001170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e087ee47-1768-404b-81f8-7a6de25ccca6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:03.001929 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087ee47-1768-404b-81f8-7a6de25ccca6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:03.003513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e087ee47-1768-404b-81f8-7a6de25ccca6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:03.014436 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1183/2348] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:17:03.018868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a152bd-4360-4f71-b41d-17130421bbe6 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:03.029156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a152bd-4360-4f71-b41d-17130421bbe6 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:03.033966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e87d7-7190-47c5-986d-e4a1ec1ed89e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:03.034853 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e87d7-7190-47c5-986d-e4a1ec1ed89e admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:03.035132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e87d7-7190-47c5-986d-e4a1ec1ed89e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:03.035410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e87d7-7190-47c5-986d-e4a1ec1ed89e admin admin] PATH_INFO: `/v3/projects/f5484157d3894ec097d1f1b2e0bacc16/users/8cd8f0490c6949cbba431b0b317683fd/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:03.051387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e87d7-7190-47c5-986d-e4a1ec1ed89e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5484157d3894ec097d1f1b2e0bacc16, user_id=8cd8f0490c6949cbba431b0b317683fd, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:03.054224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e87d7-7190-47c5-986d-e4a1ec1ed89e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:03.067843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1166/2349] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:17:03 2022] PUT /identity/v3/projects/f5484157d3894ec097d1f1b2e0bacc16/users/8cd8f0490c6949cbba431b0b317683fd/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:17:03.073679 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3a673-9800-455d-948b-636f77ad56b3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:03.074000 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3a673-9800-455d-948b-636f77ad56b3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:03.074326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3a673-9800-455d-948b-636f77ad56b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:03.092428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-42b3a673-9800-455d-948b-636f77ad56b3 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:17:03.093670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-42b3a673-9800-455d-948b-636f77ad56b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:03.101720 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:17:03.155115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1184/2350] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:17:03.164607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e87d7-7190-47c5-986d-e4a1ec1ed89e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:03.167125 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1d5e87d7-7190-47c5-986d-e4a1ec1ed89e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:03.224516 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e87d7-7190-47c5-986d-e4a1ec1ed89e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:03.228987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e9c5e-230c-41ec-b2a1-6d7e89dc0009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:03.229648 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e9c5e-230c-41ec-b2a1-6d7e89dc0009 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:03.229935 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e9c5e-230c-41ec-b2a1-6d7e89dc0009 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:03.230209 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e9c5e-230c-41ec-b2a1-6d7e89dc0009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:03.239877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e9c5e-230c-41ec-b2a1-6d7e89dc0009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:03.241342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e9c5e-230c-41ec-b2a1-6d7e89dc0009 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:03.252392 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1167/2351] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:17:07.188672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba3922-36c3-42f9-8276-84cdf6d73979 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:07.189215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba3922-36c3-42f9-8276-84cdf6d73979 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:07.189324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ba3922-36c3-42f9-8276-84cdf6d73979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:07.217448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e1ba3922-36c3-42f9-8276-84cdf6d73979 None None] MFA Rules not processed for user `8cd8f0490c6949cbba431b0b317683fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:17:07.218661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e1ba3922-36c3-42f9-8276-84cdf6d73979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:07.316441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1185/2352] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:17:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:17:07.317527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e9c5e-230c-41ec-b2a1-6d7e89dc0009 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:07.322988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:17:07.340017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e9c5e-230c-41ec-b2a1-6d7e89dc0009 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:07.349131 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc29818-2cab-4fd8-9eef-263181b07911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:07.350018 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc29818-2cab-4fd8-9eef-263181b07911 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:07.350303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc29818-2cab-4fd8-9eef-263181b07911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:07.350574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc29818-2cab-4fd8-9eef-263181b07911 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:07.351291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc29818-2cab-4fd8-9eef-263181b07911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:07.353825 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc29818-2cab-4fd8-9eef-263181b07911 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:07.384912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1168/2353] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:17:07 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:17:07.386154 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba3922-36c3-42f9-8276-84cdf6d73979 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:07.397692 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ba3922-36c3-42f9-8276-84cdf6d73979 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:07.485418 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38097c93-cd3a-4d14-93fc-5d48439ffbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:07.486493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38097c93-cd3a-4d14-93fc-5d48439ffbae admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:07.491710 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38097c93-cd3a-4d14-93fc-5d48439ffbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:07.492013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38097c93-cd3a-4d14-93fc-5d48439ffbae admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:07.492782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38097c93-cd3a-4d14-93fc-5d48439ffbae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:07.495431 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38097c93-cd3a-4d14-93fc-5d48439ffbae admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:07.537492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1186/2354] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:17:07 2022] POST /identity/v3/users => generated 295 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:17:07.542081 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc29818-2cab-4fd8-9eef-263181b07911 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:07.553953 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc29818-2cab-4fd8-9eef-263181b07911 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:07.559285 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6dd2e-f92c-4908-90d2-b8aa6149def7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:07.559894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6dd2e-f92c-4908-90d2-b8aa6149def7 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:07.560174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6dd2e-f92c-4908-90d2-b8aa6149def7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:07.560448 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6dd2e-f92c-4908-90d2-b8aa6149def7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:07.561696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6dd2e-f92c-4908-90d2-b8aa6149def7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:07.563301 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6dd2e-f92c-4908-90d2-b8aa6149def7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:07.576186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1169/2355] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:17:07.580255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38097c93-cd3a-4d14-93fc-5d48439ffbae admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:07.592019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38097c93-cd3a-4d14-93fc-5d48439ffbae admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:07.596873 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a26d0-f017-4f91-b918-0c16737b414b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:07.597818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a26d0-f017-4f91-b918-0c16737b414b admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:07.598104 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a26d0-f017-4f91-b918-0c16737b414b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:07.598384 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6a26d0-f017-4f91-b918-0c16737b414b admin admin] PATH_INFO: `/v3/projects/a7f90d9e3e7e463a97212fe1d8dca158/users/8463a3a31a4f42b18c26aefd8cbe4844/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:07.615877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a26d0-f017-4f91-b918-0c16737b414b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7f90d9e3e7e463a97212fe1d8dca158, user_id=8463a3a31a4f42b18c26aefd8cbe4844, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:07.619233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6a26d0-f017-4f91-b918-0c16737b414b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:07.633016 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1187/2356] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:17:07 2022] PUT /identity/v3/projects/a7f90d9e3e7e463a97212fe1d8dca158/users/8463a3a31a4f42b18c26aefd8cbe4844/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:17:07.637580 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6dd2e-f92c-4908-90d2-b8aa6149def7 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:07.647917 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6dd2e-f92c-4908-90d2-b8aa6149def7 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:07.709926 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2015ea-1bb7-4939-beb7-9cb5982714b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:07.710602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2015ea-1bb7-4939-beb7-9cb5982714b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:07.710879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2015ea-1bb7-4939-beb7-9cb5982714b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:07.711151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2015ea-1bb7-4939-beb7-9cb5982714b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:07.713618 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2015ea-1bb7-4939-beb7-9cb5982714b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:07.713752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2015ea-1bb7-4939-beb7-9cb5982714b5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:07.728148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1170/2357] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:17:07 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:17:07.729402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a26d0-f017-4f91-b918-0c16737b414b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:07.739954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6a26d0-f017-4f91-b918-0c16737b414b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:07.745470 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e69d1-a3b1-40b6-9e39-a034f949ee3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:07.746376 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e69d1-a3b1-40b6-9e39-a034f949ee3f admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:07.746657 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e69d1-a3b1-40b6-9e39-a034f949ee3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:07.746933 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88e69d1-a3b1-40b6-9e39-a034f949ee3f admin admin] PATH_INFO: `/v3/projects/a7f90d9e3e7e463a97212fe1d8dca158/users/8463a3a31a4f42b18c26aefd8cbe4844/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:07.748913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e69d1-a3b1-40b6-9e39-a034f949ee3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7f90d9e3e7e463a97212fe1d8dca158, user_id=8463a3a31a4f42b18c26aefd8cbe4844, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:07.751627 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88e69d1-a3b1-40b6-9e39-a034f949ee3f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:07.755804 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:17:07.771754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1188/2358] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:17:07 2022] PUT /identity/v3/projects/a7f90d9e3e7e463a97212fe1d8dca158/users/8463a3a31a4f42b18c26aefd8cbe4844/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:17:12.341878 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a6dd8-fe39-4ef8-814d-7c9b264b94f7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:12.342516 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a6dd8-fe39-4ef8-814d-7c9b264b94f7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:12.342516 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a6dd8-fe39-4ef8-814d-7c9b264b94f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:12.370667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a46a6dd8-fe39-4ef8-814d-7c9b264b94f7 None None] MFA Rules not processed for user `8463a3a31a4f42b18c26aefd8cbe4844`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:17:12.371916 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a46a6dd8-fe39-4ef8-814d-7c9b264b94f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:12.453143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1171/2359] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:17:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:17:12.460707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e69d1-a3b1-40b6-9e39-a034f949ee3f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:12.464832 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e88e69d1-a3b1-40b6-9e39-a034f949ee3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:12.564846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88e69d1-a3b1-40b6-9e39-a034f949ee3f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:12.572871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abc084c-e4e9-4bfa-85d5-df02453642b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:12.573807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc084c-e4e9-4bfa-85d5-df02453642b1 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:12.574263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc084c-e4e9-4bfa-85d5-df02453642b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:12.574608 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abc084c-e4e9-4bfa-85d5-df02453642b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:12.635880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc084c-e4e9-4bfa-85d5-df02453642b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:12.637754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abc084c-e4e9-4bfa-85d5-df02453642b1 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:12.651301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1189/2360] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:17:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:17:22.443031 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=76,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:17:22.643896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=58 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:17:30.462125 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a6dd8-fe39-4ef8-814d-7c9b264b94f7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:30.463534 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a46a6dd8-fe39-4ef8-814d-7c9b264b94f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:30.532277 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a6dd8-fe39-4ef8-814d-7c9b264b94f7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:30.536681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0f902-e340-4711-9b55-f50328373914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:30.537579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0f902-e340-4711-9b55-f50328373914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:30.537858 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0f902-e340-4711-9b55-f50328373914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:30.538129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c0f902-e340-4711-9b55-f50328373914 admin admin] PATH_INFO: `/v3/users/b612d0e2c43444b991ea53cc996591eb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:30.539314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0f902-e340-4711-9b55-f50328373914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b612d0e2c43444b991ea53cc996591eb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:30.541270 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c0f902-e340-4711-9b55-f50328373914 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:30.588060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-72c0f902-e340-4711-9b55-f50328373914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b612d0e2c43444b991ea53cc996591eb'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:30.588716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-72c0f902-e340-4711-9b55-f50328373914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b612d0e2c43444b991ea53cc996591eb'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:30.600514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-72c0f902-e340-4711-9b55-f50328373914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b612d0e2c43444b991ea53cc996591eb'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:30.607359 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-72c0f902-e340-4711-9b55-f50328373914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b612d0e2c43444b991ea53cc996591eb'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:30.614885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-72c0f902-e340-4711-9b55-f50328373914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b612d0e2c43444b991ea53cc996591eb'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:30.621502 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1172/2361] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:17:30 2022] DELETE /identity/v3/users/b612d0e2c43444b991ea53cc996591eb => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:17:30.626509 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31d5c6-5f67-4717-846f-3bc7874ae6db None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:30.626948 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31d5c6-5f67-4717-846f-3bc7874ae6db None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:30.627064 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31d5c6-5f67-4717-846f-3bc7874ae6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:30.644156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1b31d5c6-5f67-4717-846f-3bc7874ae6db None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:17:30.645371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1b31d5c6-5f67-4717-846f-3bc7874ae6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:30.706238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1190/2362] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:17:30.715891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0f902-e340-4711-9b55-f50328373914 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:30.717322 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-72c0f902-e340-4711-9b55-f50328373914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:30.776040 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c0f902-e340-4711-9b55-f50328373914 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:30.780393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def8890-b094-41aa-b83a-99a30380062d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:30.781081 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def8890-b094-41aa-b83a-99a30380062d service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:30.781390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def8890-b094-41aa-b83a-99a30380062d service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:30.781665 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def8890-b094-41aa-b83a-99a30380062d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:30.790788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def8890-b094-41aa-b83a-99a30380062d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:30.792157 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def8890-b094-41aa-b83a-99a30380062d service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:30.802205 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1173/2363] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:17:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:17:31.084123 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31d5c6-5f67-4717-846f-3bc7874ae6db None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:31.085940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1b31d5c6-5f67-4717-846f-3bc7874ae6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:31.099260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31d5c6-5f67-4717-846f-3bc7874ae6db None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:31.104375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908fe70-8ed1-477b-ac36-f7a30afb753e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:31.105823 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908fe70-8ed1-477b-ac36-f7a30afb753e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:31.106202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908fe70-8ed1-477b-ac36-f7a30afb753e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:31.106661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908fe70-8ed1-477b-ac36-f7a30afb753e admin admin] PATH_INFO: `/v3/projects/524df7e140f74c69875de81de9ed7e32` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:31.108651 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908fe70-8ed1-477b-ac36-f7a30afb753e admin admin] RBAC: Authorizing `identity:delete_project(project_id=524df7e140f74c69875de81de9ed7e32)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:31.115191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908fe70-8ed1-477b-ac36-f7a30afb753e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:31.162314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3908fe70-8ed1-477b-ac36-f7a30afb753e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '524df7e140f74c69875de81de9ed7e32', 'request_id': 'req-3908fe70-8ed1-477b-ac36-f7a30afb753e'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:31.168579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3908fe70-8ed1-477b-ac36-f7a30afb753e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '524df7e140f74c69875de81de9ed7e32', 'request_id': 'req-3908fe70-8ed1-477b-ac36-f7a30afb753e'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:31.175624 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3908fe70-8ed1-477b-ac36-f7a30afb753e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '524df7e140f74c69875de81de9ed7e32', 'request_id': 'req-3908fe70-8ed1-477b-ac36-f7a30afb753e'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:31.189749 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3908fe70-8ed1-477b-ac36-f7a30afb753e admin admin] The token cache is being invalidate because project 524df7e140f74c69875de81de9ed7e32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:17:31.190039 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3908fe70-8ed1-477b-ac36-f7a30afb753e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:31.190657 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-3908fe70-8ed1-477b-ac36-f7a30afb753e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:31.192219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1191/2364] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:17:31 2022] DELETE /identity/v3/projects/524df7e140f74c69875de81de9ed7e32 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:17:31.197270 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630f059-45f2-4cfb-969d-73c405184be5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:31.197556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630f059-45f2-4cfb-969d-73c405184be5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:31.197824 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630f059-45f2-4cfb-969d-73c405184be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:31.216829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a630f059-45f2-4cfb-969d-73c405184be5 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:17:31.218196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a630f059-45f2-4cfb-969d-73c405184be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:31.238504 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:17:31.283079 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1174/2365] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:17:31.291342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92653727-d06f-4238-b6de-215f5aaa5061 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:31.291616 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92653727-d06f-4238-b6de-215f5aaa5061 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:31.291893 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92653727-d06f-4238-b6de-215f5aaa5061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:31.311278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-92653727-d06f-4238-b6de-215f5aaa5061 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:17:31.312566 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-92653727-d06f-4238-b6de-215f5aaa5061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:31.320751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1192/2366] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:17:31.325195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630f059-45f2-4cfb-969d-73c405184be5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:31.339145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630f059-45f2-4cfb-969d-73c405184be5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:31.343736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5254a8-3e3b-4b21-bbad-4cb21bbba2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:31.344400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5254a8-3e3b-4b21-bbad-4cb21bbba2d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:31.344676 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5254a8-3e3b-4b21-bbad-4cb21bbba2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:31.344994 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5254a8-3e3b-4b21-bbad-4cb21bbba2d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:31.345728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5254a8-3e3b-4b21-bbad-4cb21bbba2d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:17:31.346035 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5254a8-3e3b-4b21-bbad-4cb21bbba2d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:31.347629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5254a8-3e3b-4b21-bbad-4cb21bbba2d6 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:31.360113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1175/2367] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:17:31.364377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92653727-d06f-4238-b6de-215f5aaa5061 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:31.374519 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92653727-d06f-4238-b6de-215f5aaa5061 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:31.379251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41174a51-18af-45fe-877b-2dce243d6f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:31.379918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41174a51-18af-45fe-877b-2dce243d6f8f admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:31.380179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41174a51-18af-45fe-877b-2dce243d6f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:31.380452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41174a51-18af-45fe-877b-2dce243d6f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:31.381189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41174a51-18af-45fe-877b-2dce243d6f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:31.383076 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41174a51-18af-45fe-877b-2dce243d6f8f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:31.411034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1193/2368] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:17:31 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:17:31.415149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5254a8-3e3b-4b21-bbad-4cb21bbba2d6 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:31.426091 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5254a8-3e3b-4b21-bbad-4cb21bbba2d6 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:31.484627 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb17a89-f6a8-4ba1-a1ac-2503dd31125a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:31.485390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb17a89-f6a8-4ba1-a1ac-2503dd31125a admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:31.485647 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb17a89-f6a8-4ba1-a1ac-2503dd31125a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:31.485919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb17a89-f6a8-4ba1-a1ac-2503dd31125a admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:31.486617 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb17a89-f6a8-4ba1-a1ac-2503dd31125a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:31.488526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb17a89-f6a8-4ba1-a1ac-2503dd31125a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:31.508274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1176/2369] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:17:31 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:17:31.512518 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41174a51-18af-45fe-877b-2dce243d6f8f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:31.523297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41174a51-18af-45fe-877b-2dce243d6f8f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:31.528183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28036f6-36b8-4c7c-8ab9-418e04cb3091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:31.528983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28036f6-36b8-4c7c-8ab9-418e04cb3091 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:31.529339 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28036f6-36b8-4c7c-8ab9-418e04cb3091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:31.529698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28036f6-36b8-4c7c-8ab9-418e04cb3091 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:31.530618 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28036f6-36b8-4c7c-8ab9-418e04cb3091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:31.532430 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28036f6-36b8-4c7c-8ab9-418e04cb3091 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:31.545510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1194/2370] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:17:31.549615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb17a89-f6a8-4ba1-a1ac-2503dd31125a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:31.559697 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb17a89-f6a8-4ba1-a1ac-2503dd31125a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:31.565116 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bcf74-fd39-401d-aeb4-65776e0147a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:31.565777 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bcf74-fd39-401d-aeb4-65776e0147a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:31.566056 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bcf74-fd39-401d-aeb4-65776e0147a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:31.566326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bcf74-fd39-401d-aeb4-65776e0147a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:31.567076 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bcf74-fd39-401d-aeb4-65776e0147a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:31.568667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bcf74-fd39-401d-aeb4-65776e0147a5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:31.579566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1177/2371] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:17:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:17:31.583747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28036f6-36b8-4c7c-8ab9-418e04cb3091 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:31.594664 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28036f6-36b8-4c7c-8ab9-418e04cb3091 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:31.599297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d82f7a-46fa-4473-a16e-9c139793042b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:31.600174 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d82f7a-46fa-4473-a16e-9c139793042b admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:31.600456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d82f7a-46fa-4473-a16e-9c139793042b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:31.600732 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d82f7a-46fa-4473-a16e-9c139793042b admin admin] PATH_INFO: `/v3/projects/84cdc3e0745d47a389f55770faff5a13/users/250734b209d944b2aab8834d5b95a563/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:31.615134 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d82f7a-46fa-4473-a16e-9c139793042b admin admin] RBAC: Authorizing `identity:create_grant(project_id=84cdc3e0745d47a389f55770faff5a13, user_id=250734b209d944b2aab8834d5b95a563, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:31.617821 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d82f7a-46fa-4473-a16e-9c139793042b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:31.630962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1195/2372] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:17:31 2022] PUT /identity/v3/projects/84cdc3e0745d47a389f55770faff5a13/users/250734b209d944b2aab8834d5b95a563/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:17:31.643647 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7a408-17ea-4917-9ed4-f7e1ab6c44f4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:31.643938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7a408-17ea-4917-9ed4-f7e1ab6c44f4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:31.644221 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7a408-17ea-4917-9ed4-f7e1ab6c44f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:31.675248 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b9f7a408-17ea-4917-9ed4-f7e1ab6c44f4 None None] MFA Rules not processed for user `250734b209d944b2aab8834d5b95a563`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:17:31.676502 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b9f7a408-17ea-4917-9ed4-f7e1ab6c44f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:31.741710 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1178/2373] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:17:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:17:31.753114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d82f7a-46fa-4473-a16e-9c139793042b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:31.754841 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-47d82f7a-46fa-4473-a16e-9c139793042b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:31.777022 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:17:31.815120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d82f7a-46fa-4473-a16e-9c139793042b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:31.818860 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b07c9-8925-4911-b92e-593b2eb601d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:31.819500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b07c9-8925-4911-b92e-593b2eb601d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:31.819783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b07c9-8925-4911-b92e-593b2eb601d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:31.820059 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b07c9-8925-4911-b92e-593b2eb601d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:31.828849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b07c9-8925-4911-b92e-593b2eb601d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:31.830360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b07c9-8925-4911-b92e-593b2eb601d3 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:31.838994 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1196/2374] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:17:31 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:17:41.744389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:17:41.835053 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=17 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:17:43.128132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7a408-17ea-4917-9ed4-f7e1ab6c44f4 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:43.188561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7a408-17ea-4917-9ed4-f7e1ab6c44f4 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:43.193123 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd978e80-a728-48b7-8036-b1051417066e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:43.193988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd978e80-a728-48b7-8036-b1051417066e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:43.194266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd978e80-a728-48b7-8036-b1051417066e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:43.194540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd978e80-a728-48b7-8036-b1051417066e admin admin] PATH_INFO: `/v3/users/250734b209d944b2aab8834d5b95a563` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:43.195728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd978e80-a728-48b7-8036-b1051417066e admin admin] RBAC: Authorizing `identity:delete_user(user_id=250734b209d944b2aab8834d5b95a563)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:43.197682 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd978e80-a728-48b7-8036-b1051417066e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:43.245254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fd978e80-a728-48b7-8036-b1051417066e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '250734b209d944b2aab8834d5b95a563'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:43.245917 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fd978e80-a728-48b7-8036-b1051417066e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '250734b209d944b2aab8834d5b95a563'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:43.259114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fd978e80-a728-48b7-8036-b1051417066e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '250734b209d944b2aab8834d5b95a563'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:43.266783 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fd978e80-a728-48b7-8036-b1051417066e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '250734b209d944b2aab8834d5b95a563'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:43.273863 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fd978e80-a728-48b7-8036-b1051417066e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '250734b209d944b2aab8834d5b95a563'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:43.281275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1179/2375] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:17:43 2022] DELETE /identity/v3/users/250734b209d944b2aab8834d5b95a563 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:17:43.285930 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d88404-1479-4c05-8253-7db7999f53be None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:43.286415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d88404-1479-4c05-8253-7db7999f53be None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:43.286524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d88404-1479-4c05-8253-7db7999f53be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:43.307572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-07d88404-1479-4c05-8253-7db7999f53be None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:17:43.309806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-07d88404-1479-4c05-8253-7db7999f53be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:43.373603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1197/2376] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:17:43.383156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd978e80-a728-48b7-8036-b1051417066e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:43.384583 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fd978e80-a728-48b7-8036-b1051417066e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:43.443265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd978e80-a728-48b7-8036-b1051417066e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:43.446908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e54d2-2da4-42e1-a584-2635e3c89480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:43.447551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e54d2-2da4-42e1-a584-2635e3c89480 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:43.447833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e54d2-2da4-42e1-a584-2635e3c89480 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:43.448109 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139e54d2-2da4-42e1-a584-2635e3c89480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:43.457573 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e54d2-2da4-42e1-a584-2635e3c89480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:43.458991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139e54d2-2da4-42e1-a584-2635e3c89480 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:43.468907 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1180/2377] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:17:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:17:43.697815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cf67b-c5c3-462a-904d-09e73ab3982d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:43.698106 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cf67b-c5c3-462a-904d-09e73ab3982d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:43.698379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cf67b-c5c3-462a-904d-09e73ab3982d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:43.715593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a91cf67b-c5c3-462a-904d-09e73ab3982d None None] MFA Rules not processed for user `8cd8f0490c6949cbba431b0b317683fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:17:43.717396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a91cf67b-c5c3-462a-904d-09e73ab3982d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:43.745845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e54d2-2da4-42e1-a584-2635e3c89480 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:17:43.747564 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-139e54d2-2da4-42e1-a584-2635e3c89480 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:43.760991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139e54d2-2da4-42e1-a584-2635e3c89480 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:17:43.766604 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176d09d-d705-4f0d-973b-d94342b0e441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:17:43.767592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176d09d-d705-4f0d-973b-d94342b0e441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:43.767871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176d09d-d705-4f0d-973b-d94342b0e441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:43.768231 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176d09d-d705-4f0d-973b-d94342b0e441 admin admin] PATH_INFO: `/v3/projects/84cdc3e0745d47a389f55770faff5a13` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:43.769490 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176d09d-d705-4f0d-973b-d94342b0e441 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84cdc3e0745d47a389f55770faff5a13)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:17:43.771407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7176d09d-d705-4f0d-973b-d94342b0e441 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:17:43.790379 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:17:43.790820 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1198/2378] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:17:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:17:43.823954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7176d09d-d705-4f0d-973b-d94342b0e441 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84cdc3e0745d47a389f55770faff5a13', 'request_id': 'req-7176d09d-d705-4f0d-973b-d94342b0e441'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:43.831801 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7176d09d-d705-4f0d-973b-d94342b0e441 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84cdc3e0745d47a389f55770faff5a13', 'request_id': 'req-7176d09d-d705-4f0d-973b-d94342b0e441'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:43.839998 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7176d09d-d705-4f0d-973b-d94342b0e441 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84cdc3e0745d47a389f55770faff5a13', 'request_id': 'req-7176d09d-d705-4f0d-973b-d94342b0e441'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:43.851575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7176d09d-d705-4f0d-973b-d94342b0e441 admin admin] The token cache is being invalidate because project 84cdc3e0745d47a389f55770faff5a13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:17:43.852010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7176d09d-d705-4f0d-973b-d94342b0e441 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:43.852782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-7176d09d-d705-4f0d-973b-d94342b0e441 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:17:43.854398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1181/2379] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:17:43 2022] DELETE /identity/v3/projects/84cdc3e0745d47a389f55770faff5a13 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:17:47.202116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da479900-9f57-4dc1-841a-04d23dd520a3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:17:47.202598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da479900-9f57-4dc1-841a-04d23dd520a3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:17:47.202698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da479900-9f57-4dc1-841a-04d23dd520a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:17:47.222918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-da479900-9f57-4dc1-841a-04d23dd520a3 None None] MFA Rules not processed for user `9eebb3ed7b2342efbd3eb4e72329c1e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:17:47.224117 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-da479900-9f57-4dc1-841a-04d23dd520a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:17:47.278991 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1199/2380] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:17:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:17:53.848445 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=17,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:17:57.271700 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=77 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:18:02.432283 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176d09d-d705-4f0d-973b-d94342b0e441 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:18:02.433766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7176d09d-d705-4f0d-973b-d94342b0e441 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:18:02.491339 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7176d09d-d705-4f0d-973b-d94342b0e441 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:18:02.494920 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0ff99-5e40-4d5c-b934-920013dc43f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:18:02.495562 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0ff99-5e40-4d5c-b934-920013dc43f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:18:02.495843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0ff99-5e40-4d5c-b934-920013dc43f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:18:02.496118 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0ff99-5e40-4d5c-b934-920013dc43f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:18:02.497993 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-19b0ff99-5e40-4d5c-b934-920013dc43f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:18:02.560579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0ff99-5e40-4d5c-b934-920013dc43f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:18:02.562051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b0ff99-5e40-4d5c-b934-920013dc43f6 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:18:02.572348 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1182/2381] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:18:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:18:12.567540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=50 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:18:29.017463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b418d9e-7c08-4f70-abd5-417342378320 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:18:29.017937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b418d9e-7c08-4f70-abd5-417342378320 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:18:29.018024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b418d9e-7c08-4f70-abd5-417342378320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:18:29.036447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5b418d9e-7c08-4f70-abd5-417342378320 None None] MFA Rules not processed for user `bff5f2d1367545f2b4c77fe0d9f095ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:18:29.037703 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5b418d9e-7c08-4f70-abd5-417342378320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:18:29.094868 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1200/2382] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:18:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:18:39.087913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=25 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:18:42.539136 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e496b0f-c86f-4868-adc6-0855c33c9a74 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:18:42.539637 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e496b0f-c86f-4868-adc6-0855c33c9a74 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:18:42.539736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e496b0f-c86f-4868-adc6-0855c33c9a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:18:42.557282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8e496b0f-c86f-4868-adc6-0855c33c9a74 None None] MFA Rules not processed for user `ca339e26746a4feaaa25b14be59479a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:18:42.558565 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8e496b0f-c86f-4868-adc6-0855c33c9a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:18:42.615861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1183/2383] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:18:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:18:45.210590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b418d9e-7c08-4f70-abd5-417342378320 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:18:45.221113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b418d9e-7c08-4f70-abd5-417342378320 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:18:45.224676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54d114-aa6c-4522-8822-d11e6084850c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:18:45.225356 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54d114-aa6c-4522-8822-d11e6084850c service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:18:45.225641 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54d114-aa6c-4522-8822-d11e6084850c service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:18:45.225914 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54d114-aa6c-4522-8822-d11e6084850c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:18:45.227701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5f54d114-aa6c-4522-8822-d11e6084850c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:18:45.307203 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54d114-aa6c-4522-8822-d11e6084850c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:18:45.309341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54d114-aa6c-4522-8822-d11e6084850c service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:18:45.321946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1201/2384] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:18:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:18:48.062315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e496b0f-c86f-4868-adc6-0855c33c9a74 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:18:48.079745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e496b0f-c86f-4868-adc6-0855c33c9a74 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:18:48.086019 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed05bbe-aced-4f75-af03-40c31c1bfa23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:18:48.086723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed05bbe-aced-4f75-af03-40c31c1bfa23 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:18:48.087010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed05bbe-aced-4f75-af03-40c31c1bfa23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:18:48.087287 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed05bbe-aced-4f75-af03-40c31c1bfa23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:18:48.089681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-aed05bbe-aced-4f75-af03-40c31c1bfa23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:18:48.103736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed05bbe-aced-4f75-af03-40c31c1bfa23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:18:48.105929 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed05bbe-aced-4f75-af03-40c31c1bfa23 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:18:48.118156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1184/2385] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:18:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:18:50.592366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54d114-aa6c-4522-8822-d11e6084850c service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:18:50.593848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5f54d114-aa6c-4522-8822-d11e6084850c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:18:50.679822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54d114-aa6c-4522-8822-d11e6084850c service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:18:50.684796 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8a444-f55c-4fe6-b3e8-2cd2329eff55 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:18:50.685471 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8a444-f55c-4fe6-b3e8-2cd2329eff55 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:18:50.686251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4f3faf-b9a0-4f21-b90a-5bc735169397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:18:50.686694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8a444-f55c-4fe6-b3e8-2cd2329eff55 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:18:50.687133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f3faf-b9a0-4f21-b90a-5bc735169397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:18:50.687410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f3faf-b9a0-4f21-b90a-5bc735169397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:18:50.687682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4f3faf-b9a0-4f21-b90a-5bc735169397 admin admin] PATH_INFO: `/v3/users/9eebb3ed7b2342efbd3eb4e72329c1e3` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:18:50.688452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1185/2386] 10.222.0.61 () {54 vars in 921 bytes} [Thu Jul 14 10:18:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:18:50.693446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f3faf-b9a0-4f21-b90a-5bc735169397 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eebb3ed7b2342efbd3eb4e72329c1e3)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:18:50.694789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4a840-dd3e-4791-ad36-eab6037c6b24 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:18:50.695391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4a840-dd3e-4791-ad36-eab6037c6b24 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:18:50.695723 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4f3faf-b9a0-4f21-b90a-5bc735169397 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:18:50.695871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc4a840-dd3e-4791-ad36-eab6037c6b24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:18:50.721512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-5dc4a840-dd3e-4791-ad36-eab6037c6b24 None None] MFA Rules not processed for user `cd6d19c4406b4b228065b98a9a074b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:18:50.722728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5dc4a840-dd3e-4791-ad36-eab6037c6b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:18:50.732064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1186/2387] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Jul 14 10:18:50 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:18:50.755915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4a840-dd3e-4791-ad36-eab6037c6b24 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:18:50.771675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc4a840-dd3e-4791-ad36-eab6037c6b24 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:18:50.775657 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1b4f3faf-b9a0-4f21-b90a-5bc735169397 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eebb3ed7b2342efbd3eb4e72329c1e3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:18:50.776962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1b4f3faf-b9a0-4f21-b90a-5bc735169397 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eebb3ed7b2342efbd3eb4e72329c1e3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:18:50.780556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3deab2-b4f0-4976-8366-cce6739e7d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:18:50.781375 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3deab2-b4f0-4976-8366-cce6739e7d15 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:18:50.781657 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3deab2-b4f0-4976-8366-cce6739e7d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:18:50.782129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3deab2-b4f0-4976-8366-cce6739e7d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:18:50.785331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2f3deab2-b4f0-4976-8366-cce6739e7d15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:18:50.797411 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1b4f3faf-b9a0-4f21-b90a-5bc735169397 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eebb3ed7b2342efbd3eb4e72329c1e3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:18:50.805724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3deab2-b4f0-4976-8366-cce6739e7d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:18:50.811817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3deab2-b4f0-4976-8366-cce6739e7d15 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:18:50.813876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1b4f3faf-b9a0-4f21-b90a-5bc735169397 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eebb3ed7b2342efbd3eb4e72329c1e3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:18:50.825846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1187/2388] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:18:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:18:50.826257 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1b4f3faf-b9a0-4f21-b90a-5bc735169397 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eebb3ed7b2342efbd3eb4e72329c1e3'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:18:50.842488 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1202/2389] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:18:50 2022] DELETE /identity/v3/users/9eebb3ed7b2342efbd3eb4e72329c1e3 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:18:50.951511 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d141cb3-95cb-43e1-8584-3d20971c56dc None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:18:50.951798 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d141cb3-95cb-43e1-8584-3d20971c56dc None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:18:50.952089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d141cb3-95cb-43e1-8584-3d20971c56dc None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:18:50.953524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1188/2390] 10.222.0.61 () {54 vars in 921 bytes} [Thu Jul 14 10:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:18:50.960256 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae76d7a-4264-461b-a72a-b565f9213bd5 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:18:50.960871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae76d7a-4264-461b-a72a-b565f9213bd5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:18:50.961382 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae76d7a-4264-461b-a72a-b565f9213bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:18:50.969772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:18:50.983241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-bae76d7a-4264-461b-a72a-b565f9213bd5 None None] MFA Rules not processed for user `e9f4efa3e2f343c2ac858a277601a7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:18:50.986117 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bae76d7a-4264-461b-a72a-b565f9213bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:18:51.061187 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1203/2391] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Jul 14 10:18:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:18:51.072455 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d141cb3-95cb-43e1-8584-3d20971c56dc None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:18:51.074208 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5d141cb3-95cb-43e1-8584-3d20971c56dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:18:51.117563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae76d7a-4264-461b-a72a-b565f9213bd5 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:18:51.120387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bae76d7a-4264-461b-a72a-b565f9213bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:18:51.152979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d141cb3-95cb-43e1-8584-3d20971c56dc None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:18:51.156621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc3c1c-1bd9-4c70-8fae-42ec835ca6b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:18:51.157299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc3c1c-1bd9-4c70-8fae-42ec835ca6b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:18:51.157585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc3c1c-1bd9-4c70-8fae-42ec835ca6b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:18:51.157866 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc3c1c-1bd9-4c70-8fae-42ec835ca6b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:18:51.169185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc3c1c-1bd9-4c70-8fae-42ec835ca6b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:18:51.170780 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfc3c1c-1bd9-4c70-8fae-42ec835ca6b3 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:18:51.181587 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1189/2392] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:18:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:18:51.190679 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae76d7a-4264-461b-a72a-b565f9213bd5 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:18:51.195343 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d8123-06d4-41c3-a586-2df2cac537a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:18:51.196210 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d8123-06d4-41c3-a586-2df2cac537a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:18:51.196493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d8123-06d4-41c3-a586-2df2cac537a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:18:51.196766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d8123-06d4-41c3-a586-2df2cac537a9 admin admin] PATH_INFO: `/v3/projects/a2eeff0739c1463383b618c4c56ae344` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:18:51.197984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d8123-06d4-41c3-a586-2df2cac537a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2eeff0739c1463383b618c4c56ae344)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:18:51.199956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d8123-06d4-41c3-a586-2df2cac537a9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:18:51.259416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-231d8123-06d4-41c3-a586-2df2cac537a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2eeff0739c1463383b618c4c56ae344', 'request_id': 'req-231d8123-06d4-41c3-a586-2df2cac537a9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:18:51.265797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-231d8123-06d4-41c3-a586-2df2cac537a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2eeff0739c1463383b618c4c56ae344', 'request_id': 'req-231d8123-06d4-41c3-a586-2df2cac537a9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:18:51.271751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-231d8123-06d4-41c3-a586-2df2cac537a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2eeff0739c1463383b618c4c56ae344', 'request_id': 'req-231d8123-06d4-41c3-a586-2df2cac537a9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:18:51.285441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-231d8123-06d4-41c3-a586-2df2cac537a9 admin admin] The token cache is being invalidate because project a2eeff0739c1463383b618c4c56ae344 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:18:51.285746 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-231d8123-06d4-41c3-a586-2df2cac537a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:18:51.286500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-231d8123-06d4-41c3-a586-2df2cac537a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:18:51.288065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1204/2393] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:18:51 2022] DELETE /identity/v3/projects/a2eeff0739c1463383b618c4c56ae344 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:19:01.177321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=66 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:19:01.282198 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:19:20.356130 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc3c1c-1bd9-4c70-8fae-42ec835ca6b3 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:20.357594 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5dfc3c1c-1bd9-4c70-8fae-42ec835ca6b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:20.418667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc3c1c-1bd9-4c70-8fae-42ec835ca6b3 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:20.465413 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4c0436-bc3d-4b78-adf4-d0244358d579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:20.466057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c0436-bc3d-4b78-adf4-d0244358d579 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:20.466338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c0436-bc3d-4b78-adf4-d0244358d579 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:20.466611 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4c0436-bc3d-4b78-adf4-d0244358d579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:20.469056 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8e4c0436-bc3d-4b78-adf4-d0244358d579 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:20.526109 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c0436-bc3d-4b78-adf4-d0244358d579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:20.527551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4c0436-bc3d-4b78-adf4-d0244358d579 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:20.536646 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1190/2394] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:19:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:19:24.212331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d8123-06d4-41c3-a586-2df2cac537a9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:24.214358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-231d8123-06d4-41c3-a586-2df2cac537a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:24.327652 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d8123-06d4-41c3-a586-2df2cac537a9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:24.335125 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0655185-d507-4cd7-b4d4-186ce4fb0279 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:24.336017 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0655185-d507-4cd7-b4d4-186ce4fb0279 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:24.336299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0655185-d507-4cd7-b4d4-186ce4fb0279 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:24.336572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0655185-d507-4cd7-b4d4-186ce4fb0279 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:24.340135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e0655185-d507-4cd7-b4d4-186ce4fb0279 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:24.356447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0655185-d507-4cd7-b4d4-186ce4fb0279 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:24.358997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0655185-d507-4cd7-b4d4-186ce4fb0279 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:24.401530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1205/2395] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:19:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:19:27.560355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f38d5-ef22-458a-ba89-940e2a091532 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:27.560845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f38d5-ef22-458a-ba89-940e2a091532 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:27.560979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6f38d5-ef22-458a-ba89-940e2a091532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:27.578258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0b6f38d5-ef22-458a-ba89-940e2a091532 None None] MFA Rules not processed for user `01a9eebc1c8540daa31109cfc86d5ed1`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:19:27.579450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0b6f38d5-ef22-458a-ba89-940e2a091532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:27.586253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1191/2396] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:19:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:19:27.880362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0655185-d507-4cd7-b4d4-186ce4fb0279 service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:27.881873 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e0655185-d507-4cd7-b4d4-186ce4fb0279 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:27.936489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0655185-d507-4cd7-b4d4-186ce4fb0279 service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:27.940061 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad02ca-9bbe-4890-8225-03db97b016a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:27.940704 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad02ca-9bbe-4890-8225-03db97b016a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:27.941013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad02ca-9bbe-4890-8225-03db97b016a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:27.941293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad02ca-9bbe-4890-8225-03db97b016a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:27.943048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-72ad02ca-9bbe-4890-8225-03db97b016a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:27.952663 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad02ca-9bbe-4890-8225-03db97b016a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:27.954114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad02ca-9bbe-4890-8225-03db97b016a4 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:27.961912 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1206/2397] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:19:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:19:28.499848 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f38d5-ef22-458a-ba89-940e2a091532 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:28.501525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0b6f38d5-ef22-458a-ba89-940e2a091532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:28.571818 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6f38d5-ef22-458a-ba89-940e2a091532 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:28.577026 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d479d-b925-411a-92e1-140b8af77d9b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff4c42a4f4f549378e5bc1cd92f801c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:28.577673 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d479d-b925-411a-92e1-140b8af77d9b service placement] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:28.577956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d479d-b925-411a-92e1-140b8af77d9b service placement] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:28.578254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52d479d-b925-411a-92e1-140b8af77d9b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:28.580023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a52d479d-b925-411a-92e1-140b8af77d9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:28.587905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:19:28.592236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d479d-b925-411a-92e1-140b8af77d9b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:28.593835 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52d479d-b925-411a-92e1-140b8af77d9b service placement] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:28.603558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1192/2398] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:19:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:19:30.076190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad02ca-9bbe-4890-8225-03db97b016a4 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:30.077684 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-72ad02ca-9bbe-4890-8225-03db97b016a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:30.089421 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad02ca-9bbe-4890-8225-03db97b016a4 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:30.093984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf7cad-bcc3-4f1e-8e7a-0aa1530e0de7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:30.094638 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf7cad-bcc3-4f1e-8e7a-0aa1530e0de7 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:30.094917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf7cad-bcc3-4f1e-8e7a-0aa1530e0de7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:30.095191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf7cad-bcc3-4f1e-8e7a-0aa1530e0de7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:30.097013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-35bf7cad-bcc3-4f1e-8e7a-0aa1530e0de7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:30.158497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf7cad-bcc3-4f1e-8e7a-0aa1530e0de7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:30.159921 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bf7cad-bcc3-4f1e-8e7a-0aa1530e0de7 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:30.169103 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1207/2399] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:19:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:19:30.443332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d479d-b925-411a-92e1-140b8af77d9b service placement] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:30.444850 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a52d479d-b925-411a-92e1-140b8af77d9b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:30.461039 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52d479d-b925-411a-92e1-140b8af77d9b service placement] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:30.465293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5616f4-bd68-41be-8170-9ce42d668145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:30.466466 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5616f4-bd68-41be-8170-9ce42d668145 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:30.467665 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5616f4-bd68-41be-8170-9ce42d668145 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:30.467881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5616f4-bd68-41be-8170-9ce42d668145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:30.470128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4f5616f4-bd68-41be-8170-9ce42d668145 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:30.481362 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5616f4-bd68-41be-8170-9ce42d668145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:30.483446 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5616f4-bd68-41be-8170-9ce42d668145 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:30.492968 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1193/2400] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:19:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:19:33.594464 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf7cad-bcc3-4f1e-8e7a-0aa1530e0de7 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:33.606620 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf7cad-bcc3-4f1e-8e7a-0aa1530e0de7 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:33.612685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bf1c2-700c-4072-8822-a6796f425363 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:33.613355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bf1c2-700c-4072-8822-a6796f425363 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:33.613637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bf1c2-700c-4072-8822-a6796f425363 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:33.613912 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14bf1c2-700c-4072-8822-a6796f425363 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:33.615652 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f14bf1c2-700c-4072-8822-a6796f425363 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:33.641660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:19:33.674458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bf1c2-700c-4072-8822-a6796f425363 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:33.676032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14bf1c2-700c-4072-8822-a6796f425363 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:33.684954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1208/2401] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:19:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:19:37.230424 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5616f4-bd68-41be-8170-9ce42d668145 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:37.254103 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5616f4-bd68-41be-8170-9ce42d668145 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:37.260723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffbc662-71cf-4e6e-9ab3-a9d240074274 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:37.261790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbc662-71cf-4e6e-9ab3-a9d240074274 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:37.262252 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbc662-71cf-4e6e-9ab3-a9d240074274 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:37.262923 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffbc662-71cf-4e6e-9ab3-a9d240074274 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:37.265226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8ffbc662-71cf-4e6e-9ab3-a9d240074274 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:37.283615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbc662-71cf-4e6e-9ab3-a9d240074274 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:37.285275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffbc662-71cf-4e6e-9ab3-a9d240074274 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:37.298990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1194/2402] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:19:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:19:43.680813 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=16 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:19:47.292291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=15 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:19:52.353450 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bf1c2-700c-4072-8822-a6796f425363 service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:52.366513 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14bf1c2-700c-4072-8822-a6796f425363 service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:52.370528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fe955-eab0-44bd-ad07-5f08bf20cc7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:52.371202 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fe955-eab0-44bd-ad07-5f08bf20cc7f service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:52.371482 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fe955-eab0-44bd-ad07-5f08bf20cc7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:52.371958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673fe955-eab0-44bd-ad07-5f08bf20cc7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:52.374901 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-673fe955-eab0-44bd-ad07-5f08bf20cc7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:52.451354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fe955-eab0-44bd-ad07-5f08bf20cc7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:52.453386 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673fe955-eab0-44bd-ad07-5f08bf20cc7f service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:52.464160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1209/2403] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:19:56.059163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f81d0-df3f-4f51-8c58-446e2c3b811d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:56.059709 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f81d0-df3f-4f51-8c58-446e2c3b811d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:56.059943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f81d0-df3f-4f51-8c58-446e2c3b811d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:56.079230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c26f81d0-df3f-4f51-8c58-446e2c3b811d None None] MFA Rules not processed for user `bff5f2d1367545f2b4c77fe0d9f095ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:19:56.081634 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c26f81d0-df3f-4f51-8c58-446e2c3b811d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:56.090540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1195/2404] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:19:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:19:56.769970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fe955-eab0-44bd-ad07-5f08bf20cc7f service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:56.772827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-673fe955-eab0-44bd-ad07-5f08bf20cc7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:56.791033 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673fe955-eab0-44bd-ad07-5f08bf20cc7f service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:56.797709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7506f9a-9d96-4af1-9e5e-68f1ec5d9580 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff4c42a4f4f549378e5bc1cd92f801c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:56.798426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7506f9a-9d96-4af1-9e5e-68f1ec5d9580 service placement] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:56.798779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7506f9a-9d96-4af1-9e5e-68f1ec5d9580 service placement] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:56.799120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7506f9a-9d96-4af1-9e5e-68f1ec5d9580 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:56.800994 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b7506f9a-9d96-4af1-9e5e-68f1ec5d9580 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:56.816233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7506f9a-9d96-4af1-9e5e-68f1ec5d9580 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:56.818491 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7506f9a-9d96-4af1-9e5e-68f1ec5d9580 service placement] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:56.841360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1210/2405] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:19:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:19:56.856004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f81d0-df3f-4f51-8c58-446e2c3b811d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:56.857481 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c26f81d0-df3f-4f51-8c58-446e2c3b811d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:56.882406 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f81d0-df3f-4f51-8c58-446e2c3b811d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:56.888402 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d776d1b1-af6d-49a7-996d-84d167b129a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:56.890054 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776d1b1-af6d-49a7-996d-84d167b129a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:56.890614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776d1b1-af6d-49a7-996d-84d167b129a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:56.890771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d776d1b1-af6d-49a7-996d-84d167b129a8 admin admin] PATH_INFO: `/v3/users/ca339e26746a4feaaa25b14be59479a4` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:56.892082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776d1b1-af6d-49a7-996d-84d167b129a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca339e26746a4feaaa25b14be59479a4)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:56.894101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d776d1b1-af6d-49a7-996d-84d167b129a8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:56.947047 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d776d1b1-af6d-49a7-996d-84d167b129a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca339e26746a4feaaa25b14be59479a4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:19:56.947714 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d776d1b1-af6d-49a7-996d-84d167b129a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca339e26746a4feaaa25b14be59479a4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:19:56.960810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d776d1b1-af6d-49a7-996d-84d167b129a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca339e26746a4feaaa25b14be59479a4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:19:56.970518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d776d1b1-af6d-49a7-996d-84d167b129a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca339e26746a4feaaa25b14be59479a4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:19:56.977770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-d776d1b1-af6d-49a7-996d-84d167b129a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca339e26746a4feaaa25b14be59479a4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:19:56.984589 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1196/2406] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:19:56 2022] DELETE /identity/v3/users/ca339e26746a4feaaa25b14be59479a4 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:19:57.171265 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7506f9a-9d96-4af1-9e5e-68f1ec5d9580 service placement] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:57.191291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7506f9a-9d96-4af1-9e5e-68f1ec5d9580 service placement] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:57.263554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581ed6ba-5569-4c0f-a08c-e9aee8a158bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:57.265090 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ed6ba-5569-4c0f-a08c-e9aee8a158bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:57.265372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ed6ba-5569-4c0f-a08c-e9aee8a158bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:57.265642 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581ed6ba-5569-4c0f-a08c-e9aee8a158bf admin admin] PATH_INFO: `/v3/projects/76e4caaf5b9e49678bc6427b71560f19` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:57.266831 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ed6ba-5569-4c0f-a08c-e9aee8a158bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=76e4caaf5b9e49678bc6427b71560f19)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:57.268780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581ed6ba-5569-4c0f-a08c-e9aee8a158bf admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:57.317692 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-581ed6ba-5569-4c0f-a08c-e9aee8a158bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76e4caaf5b9e49678bc6427b71560f19', 'request_id': 'req-581ed6ba-5569-4c0f-a08c-e9aee8a158bf'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:19:57.324127 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-581ed6ba-5569-4c0f-a08c-e9aee8a158bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76e4caaf5b9e49678bc6427b71560f19', 'request_id': 'req-581ed6ba-5569-4c0f-a08c-e9aee8a158bf'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:19:57.330274 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-581ed6ba-5569-4c0f-a08c-e9aee8a158bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76e4caaf5b9e49678bc6427b71560f19', 'request_id': 'req-581ed6ba-5569-4c0f-a08c-e9aee8a158bf'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:19:57.343215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-581ed6ba-5569-4c0f-a08c-e9aee8a158bf admin admin] The token cache is being invalidate because project 76e4caaf5b9e49678bc6427b71560f19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:19:57.343504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-581ed6ba-5569-4c0f-a08c-e9aee8a158bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:19:57.344130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-581ed6ba-5569-4c0f-a08c-e9aee8a158bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:19:57.345718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1211/2407] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:19:57 2022] DELETE /identity/v3/projects/76e4caaf5b9e49678bc6427b71560f19 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:19:57.350636 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fb40e-79da-4e59-84cc-f381c775fed2 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:57.351149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fb40e-79da-4e59-84cc-f381c775fed2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:57.351264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0fb40e-79da-4e59-84cc-f381c775fed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:57.369966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ad0fb40e-79da-4e59-84cc-f381c775fed2 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:19:57.371170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ad0fb40e-79da-4e59-84cc-f381c775fed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:57.429394 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1197/2408] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:19:57.437707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ad415-48d7-479a-9a69-c8b7cc99e321 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:57.437998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ad415-48d7-479a-9a69-c8b7cc99e321 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:57.438278 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68ad415-48d7-479a-9a69-c8b7cc99e321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:57.456783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c68ad415-48d7-479a-9a69-c8b7cc99e321 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:19:57.458010 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c68ad415-48d7-479a-9a69-c8b7cc99e321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:57.465840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1212/2409] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:19:57.470073 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0fb40e-79da-4e59-84cc-f381c775fed2 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:57.479810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0fb40e-79da-4e59-84cc-f381c775fed2 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:57.484492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4832d-e87c-479f-908c-0e2b6e70b20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:57.485177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4832d-e87c-479f-908c-0e2b6e70b20b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:57.485463 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4832d-e87c-479f-908c-0e2b6e70b20b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:57.485738 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db4832d-e87c-479f-908c-0e2b6e70b20b admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:57.486468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4832d-e87c-479f-908c-0e2b6e70b20b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:19:57.486778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4832d-e87c-479f-908c-0e2b6e70b20b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:57.488368 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db4832d-e87c-479f-908c-0e2b6e70b20b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:57.500495 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1198/2410] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:19:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:19:57.504872 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ad415-48d7-479a-9a69-c8b7cc99e321 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:57.510481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:19:57.515107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68ad415-48d7-479a-9a69-c8b7cc99e321 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:57.521147 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d1d63-1f51-4717-a59f-587a6f43abb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:57.522037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d1d63-1f51-4717-a59f-587a6f43abb8 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:57.522480 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d1d63-1f51-4717-a59f-587a6f43abb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:57.522801 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9d1d63-1f51-4717-a59f-587a6f43abb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:57.523665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d1d63-1f51-4717-a59f-587a6f43abb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:57.525607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9d1d63-1f51-4717-a59f-587a6f43abb8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:57.553905 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1213/2411] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:19:57 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:19:57.555071 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4832d-e87c-479f-908c-0e2b6e70b20b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:57.565249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db4832d-e87c-479f-908c-0e2b6e70b20b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:57.612941 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:19:57.623320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66f370-f50c-48e3-8f36-69a9408d5e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:57.624469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66f370-f50c-48e3-8f36-69a9408d5e8b admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:57.624751 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66f370-f50c-48e3-8f36-69a9408d5e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:57.625049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66f370-f50c-48e3-8f36-69a9408d5e8b admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:57.625774 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66f370-f50c-48e3-8f36-69a9408d5e8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:57.627725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66f370-f50c-48e3-8f36-69a9408d5e8b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:57.651354 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1199/2412] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:19:57 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:19:57.652638 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d1d63-1f51-4717-a59f-587a6f43abb8 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:57.663530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9d1d63-1f51-4717-a59f-587a6f43abb8 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:57.668141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c4f26-cf7c-4267-9f2a-faf7cfd1ea0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:57.668800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c4f26-cf7c-4267-9f2a-faf7cfd1ea0f admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:57.669107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c4f26-cf7c-4267-9f2a-faf7cfd1ea0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:57.669377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c4f26-cf7c-4267-9f2a-faf7cfd1ea0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:57.670214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c4f26-cf7c-4267-9f2a-faf7cfd1ea0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:57.671855 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c4f26-cf7c-4267-9f2a-faf7cfd1ea0f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:57.682882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1214/2413] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:19:57.686999 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66f370-f50c-48e3-8f36-69a9408d5e8b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:57.698523 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66f370-f50c-48e3-8f36-69a9408d5e8b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:57.704565 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f4da9-6f5f-4dfd-a71f-28e2a245c26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:57.705390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f4da9-6f5f-4dfd-a71f-28e2a245c26f admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:57.706134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f4da9-6f5f-4dfd-a71f-28e2a245c26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:57.706134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f4da9-6f5f-4dfd-a71f-28e2a245c26f admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:57.706758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f4da9-6f5f-4dfd-a71f-28e2a245c26f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:57.708369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f4da9-6f5f-4dfd-a71f-28e2a245c26f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:57.723249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1200/2414] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:19:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:19:57.727113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c4f26-cf7c-4267-9f2a-faf7cfd1ea0f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:57.740272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c4f26-cf7c-4267-9f2a-faf7cfd1ea0f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:57.746249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e99fbd-cb50-47d8-872f-b6ef8baf4f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:57.747429 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e99fbd-cb50-47d8-872f-b6ef8baf4f0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:57.747577 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e99fbd-cb50-47d8-872f-b6ef8baf4f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:57.747834 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e99fbd-cb50-47d8-872f-b6ef8baf4f0a admin admin] PATH_INFO: `/v3/projects/4f945783031448849dc0e5a3337f8c76/users/7b0f433eba57485cbbfeab651a8e8b02/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:57.765818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e99fbd-cb50-47d8-872f-b6ef8baf4f0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f945783031448849dc0e5a3337f8c76, user_id=7b0f433eba57485cbbfeab651a8e8b02, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:57.768416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e99fbd-cb50-47d8-872f-b6ef8baf4f0a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:57.784331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1215/2415] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:19:57 2022] PUT /identity/v3/projects/4f945783031448849dc0e5a3337f8c76/users/7b0f433eba57485cbbfeab651a8e8b02/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:19:57.790166 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614e89d-044c-4db0-b06c-8d87610f68e6 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:57.790623 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614e89d-044c-4db0-b06c-8d87610f68e6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:57.791087 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614e89d-044c-4db0-b06c-8d87610f68e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:57.810575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7614e89d-044c-4db0-b06c-8d87610f68e6 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:19:57.812469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7614e89d-044c-4db0-b06c-8d87610f68e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:57.878240 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1201/2416] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:19:57.888546 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e99fbd-cb50-47d8-872f-b6ef8baf4f0a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:19:57.890945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-66e99fbd-cb50-47d8-872f-b6ef8baf4f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:19:57.952230 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e99fbd-cb50-47d8-872f-b6ef8baf4f0a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:19:57.955873 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bde050-77af-4ded-bdd5-b62381485210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:19:57.956526 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bde050-77af-4ded-bdd5-b62381485210 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:19:57.956807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bde050-77af-4ded-bdd5-b62381485210 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:19:57.957112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bde050-77af-4ded-bdd5-b62381485210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:19:57.973260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bde050-77af-4ded-bdd5-b62381485210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:19:57.974666 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bde050-77af-4ded-bdd5-b62381485210 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:19:57.989498 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1216/2417] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:19:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:20:02.827467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea707fe5-e97d-47f0-84f5-73467813addf None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:20:02.827993 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea707fe5-e97d-47f0-84f5-73467813addf None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:20:02.827993 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea707fe5-e97d-47f0-84f5-73467813addf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:20:02.861082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ea707fe5-e97d-47f0-84f5-73467813addf None None] MFA Rules not processed for user `7b0f433eba57485cbbfeab651a8e8b02`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:20:02.862353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ea707fe5-e97d-47f0-84f5-73467813addf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:02.998415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1202/2418] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:20:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:20:03.011051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bde050-77af-4ded-bdd5-b62381485210 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:20:03.012657 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-60bde050-77af-4ded-bdd5-b62381485210 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:03.112764 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bde050-77af-4ded-bdd5-b62381485210 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:20:03.118413 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea38a6-4325-41ba-a11e-79a518cac51d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:20:03.119117 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea38a6-4325-41ba-a11e-79a518cac51d service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:20:03.119458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea38a6-4325-41ba-a11e-79a518cac51d service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:20:03.119661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ea38a6-4325-41ba-a11e-79a518cac51d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:20:03.129317 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea38a6-4325-41ba-a11e-79a518cac51d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:20:03.131022 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ea38a6-4325-41ba-a11e-79a518cac51d service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:20:03.141808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1217/2419] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:20:03 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:20:04.567895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11af556-7682-4142-901d-37230dce1bf1 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:20:04.568482 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11af556-7682-4142-901d-37230dce1bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:20:04.568664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11af556-7682-4142-901d-37230dce1bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:20:04.590647 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-d11af556-7682-4142-901d-37230dce1bf1 None None] MFA Rules not processed for user `7b0f433eba57485cbbfeab651a8e8b02`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:20:04.592321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d11af556-7682-4142-901d-37230dce1bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:04.603481 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1203/2420] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:20:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:20:09.470516 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea38a6-4325-41ba-a11e-79a518cac51d service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:20:09.471966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-82ea38a6-4325-41ba-a11e-79a518cac51d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:09.488114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ea38a6-4325-41ba-a11e-79a518cac51d service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:20:09.493640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02610e35-22ed-48dd-ac8e-f5c48d1daa40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:20:09.494284 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02610e35-22ed-48dd-ac8e-f5c48d1daa40 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:20:09.494634 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02610e35-22ed-48dd-ac8e-f5c48d1daa40 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:20:09.494926 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02610e35-22ed-48dd-ac8e-f5c48d1daa40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:20:09.496778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-02610e35-22ed-48dd-ac8e-f5c48d1daa40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:09.509962 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02610e35-22ed-48dd-ac8e-f5c48d1daa40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:20:09.511573 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02610e35-22ed-48dd-ac8e-f5c48d1daa40 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:20:09.522705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1218/2421] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:20:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:20:14.582905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=72,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:20:19.518001 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=88,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:20:25.814510 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f34b7d-8f42-4af8-9681-341474bcfc50 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:20:25.815055 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f34b7d-8f42-4af8-9681-341474bcfc50 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:20:25.815156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f34b7d-8f42-4af8-9681-341474bcfc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:20:25.835710 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-51f34b7d-8f42-4af8-9681-341474bcfc50 None None] MFA Rules not processed for user `8cd8f0490c6949cbba431b0b317683fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:20:25.837021 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-51f34b7d-8f42-4af8-9681-341474bcfc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:25.898353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1204/2422] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:20:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:20:32.027815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02610e35-22ed-48dd-ac8e-f5c48d1daa40 service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:20:32.037242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02610e35-22ed-48dd-ac8e-f5c48d1daa40 service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:20:32.040821 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e870e93e-691d-4a77-aef3-2b55a8588aa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:20:32.041485 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870e93e-691d-4a77-aef3-2b55a8588aa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:20:32.041768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870e93e-691d-4a77-aef3-2b55a8588aa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:20:32.042045 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e870e93e-691d-4a77-aef3-2b55a8588aa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:20:32.043888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e870e93e-691d-4a77-aef3-2b55a8588aa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:32.102825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870e93e-691d-4a77-aef3-2b55a8588aa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:20:32.104254 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e870e93e-691d-4a77-aef3-2b55a8588aa5 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:20:32.113889 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1219/2423] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:20:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:20:35.890886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=25 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:20:37.306819 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f34b7d-8f42-4af8-9681-341474bcfc50 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:20:37.308224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-51f34b7d-8f42-4af8-9681-341474bcfc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:37.320821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f34b7d-8f42-4af8-9681-341474bcfc50 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:20:37.324441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef30611-d46f-4cf3-854c-1f01de8459a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:20:37.325263 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef30611-d46f-4cf3-854c-1f01de8459a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:20:37.325545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef30611-d46f-4cf3-854c-1f01de8459a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:20:37.325840 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef30611-d46f-4cf3-854c-1f01de8459a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:20:37.327576 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7ef30611-d46f-4cf3-854c-1f01de8459a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:37.339031 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef30611-d46f-4cf3-854c-1f01de8459a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:20:37.341067 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef30611-d46f-4cf3-854c-1f01de8459a5 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:20:37.350865 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1205/2424] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:20:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:20:40.505934 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b042aa0-9eca-4b98-a62a-7e1bea200359 None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:20:40.506462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b042aa0-9eca-4b98-a62a-7e1bea200359 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:20:40.506462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b042aa0-9eca-4b98-a62a-7e1bea200359 None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:20:40.507808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1220/2425] 10.222.0.61 () {54 vars in 921 bytes} [Thu Jul 14 10:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:20:40.539560 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c0009-d5cc-4021-8086-e4ad57ff8317 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:20:40.540060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c0009-d5cc-4021-8086-e4ad57ff8317 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:20:40.540190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827c0009-d5cc-4021-8086-e4ad57ff8317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:20:40.562708 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-827c0009-d5cc-4021-8086-e4ad57ff8317 None None] MFA Rules not processed for user `ff4c42a4f4f549378e5bc1cd92f801c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:20:40.564118 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-827c0009-d5cc-4021-8086-e4ad57ff8317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:40.635278 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1206/2426] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Jul 14 10:20:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:20:40.647410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b042aa0-9eca-4b98-a62a-7e1bea200359 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:20:40.649290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4b042aa0-9eca-4b98-a62a-7e1bea200359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:40.666347 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b042aa0-9eca-4b98-a62a-7e1bea200359 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:20:40.671918 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5f674-77f4-4bf1-b993-458c30199d6f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff4c42a4f4f549378e5bc1cd92f801c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:20:40.672554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5f674-77f4-4bf1-b993-458c30199d6f service placement] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:20:40.672832 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5f674-77f4-4bf1-b993-458c30199d6f service placement] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:20:40.673167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f5f674-77f4-4bf1-b993-458c30199d6f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:20:40.681688 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5f674-77f4-4bf1-b993-458c30199d6f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:20:40.683269 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f5f674-77f4-4bf1-b993-458c30199d6f service placement] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:20:40.693625 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1221/2427] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:20:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:20:43.857948 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c0009-d5cc-4021-8086-e4ad57ff8317 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:20:43.859495 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-827c0009-d5cc-4021-8086-e4ad57ff8317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:43.874275 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827c0009-d5cc-4021-8086-e4ad57ff8317 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:20:43.878708 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2649f4-fe81-46cd-8cd2-cfeb7b30c86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:20:43.879573 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2649f4-fe81-46cd-8cd2-cfeb7b30c86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:20:43.879860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2649f4-fe81-46cd-8cd2-cfeb7b30c86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:20:43.880134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2649f4-fe81-46cd-8cd2-cfeb7b30c86b admin admin] PATH_INFO: `/v3/users/3f37d419e37b4172b27649775a144ca6` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:20:43.881344 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2649f4-fe81-46cd-8cd2-cfeb7b30c86b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f37d419e37b4172b27649775a144ca6)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:20:43.883300 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2649f4-fe81-46cd-8cd2-cfeb7b30c86b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:20:43.927674 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4d2649f4-fe81-46cd-8cd2-cfeb7b30c86b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f37d419e37b4172b27649775a144ca6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:43.928326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4d2649f4-fe81-46cd-8cd2-cfeb7b30c86b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f37d419e37b4172b27649775a144ca6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:43.946758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4d2649f4-fe81-46cd-8cd2-cfeb7b30c86b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f37d419e37b4172b27649775a144ca6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:43.954326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4d2649f4-fe81-46cd-8cd2-cfeb7b30c86b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f37d419e37b4172b27649775a144ca6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:43.962289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-4d2649f4-fe81-46cd-8cd2-cfeb7b30c86b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f37d419e37b4172b27649775a144ca6'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:43.969179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1207/2428] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:20:43 2022] DELETE /identity/v3/users/3f37d419e37b4172b27649775a144ca6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:20:44.178762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5f674-77f4-4bf1-b993-458c30199d6f service placement] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:20:44.196414 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f5f674-77f4-4bf1-b993-458c30199d6f service placement] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:20:44.293682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6945b-89c8-4723-b775-1499353e6064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:20:44.294646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6945b-89c8-4723-b775-1499353e6064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:20:44.294996 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6945b-89c8-4723-b775-1499353e6064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:20:44.295344 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6945b-89c8-4723-b775-1499353e6064 admin admin] PATH_INFO: `/v3/projects/da859b6d410b4c2b995804fa5ffd190f` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:20:44.296650 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6945b-89c8-4723-b775-1499353e6064 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da859b6d410b4c2b995804fa5ffd190f)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:20:44.298903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6945b-89c8-4723-b775-1499353e6064 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:20:44.350886 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dfb6945b-89c8-4723-b775-1499353e6064 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da859b6d410b4c2b995804fa5ffd190f', 'request_id': 'req-dfb6945b-89c8-4723-b775-1499353e6064'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:44.358814 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dfb6945b-89c8-4723-b775-1499353e6064 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da859b6d410b4c2b995804fa5ffd190f', 'request_id': 'req-dfb6945b-89c8-4723-b775-1499353e6064'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:44.364330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dfb6945b-89c8-4723-b775-1499353e6064 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da859b6d410b4c2b995804fa5ffd190f', 'request_id': 'req-dfb6945b-89c8-4723-b775-1499353e6064'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:44.377243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dfb6945b-89c8-4723-b775-1499353e6064 admin admin] The token cache is being invalidate because project da859b6d410b4c2b995804fa5ffd190f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:20:44.377536 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dfb6945b-89c8-4723-b775-1499353e6064 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:44.378174 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-dfb6945b-89c8-4723-b775-1499353e6064 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:44.379745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1222/2429] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:20:44 2022] DELETE /identity/v3/projects/da859b6d410b4c2b995804fa5ffd190f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:20:44.383710 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2649f4-fe81-46cd-8cd2-cfeb7b30c86b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:20:44.385250 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4d2649f4-fe81-46cd-8cd2-cfeb7b30c86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:44.399330 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2649f4-fe81-46cd-8cd2-cfeb7b30c86b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:20:44.450695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:20:44.459005 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51803b-ab6f-40b4-b8e2-2d9ffc71ef7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:20:44.460097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51803b-ab6f-40b4-b8e2-2d9ffc71ef7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:20:44.460581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51803b-ab6f-40b4-b8e2-2d9ffc71ef7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:20:44.461072 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51803b-ab6f-40b4-b8e2-2d9ffc71ef7a admin admin] PATH_INFO: `/v3/users/4faa600a6acd4fb7bc201806e5528037` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:20:44.462493 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51803b-ab6f-40b4-b8e2-2d9ffc71ef7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4faa600a6acd4fb7bc201806e5528037)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:20:44.464468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac51803b-ab6f-40b4-b8e2-2d9ffc71ef7a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:20:44.511476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ac51803b-ab6f-40b4-b8e2-2d9ffc71ef7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4faa600a6acd4fb7bc201806e5528037'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:44.512790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ac51803b-ab6f-40b4-b8e2-2d9ffc71ef7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4faa600a6acd4fb7bc201806e5528037'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:44.525908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ac51803b-ab6f-40b4-b8e2-2d9ffc71ef7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4faa600a6acd4fb7bc201806e5528037'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:44.533271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ac51803b-ab6f-40b4-b8e2-2d9ffc71ef7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4faa600a6acd4fb7bc201806e5528037'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:44.541010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-ac51803b-ab6f-40b4-b8e2-2d9ffc71ef7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4faa600a6acd4fb7bc201806e5528037'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:44.547883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1208/2430] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:20:44 2022] DELETE /identity/v3/users/4faa600a6acd4fb7bc201806e5528037 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:20:44.729228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6945b-89c8-4723-b775-1499353e6064 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:20:44.731252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-dfb6945b-89c8-4723-b775-1499353e6064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:44.749637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6945b-89c8-4723-b775-1499353e6064 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:20:44.763039 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:20:44.811111 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7ff35-38f4-49d2-8804-6421ee09aa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:20:44.811977 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7ff35-38f4-49d2-8804-6421ee09aa35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:20:44.812263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7ff35-38f4-49d2-8804-6421ee09aa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:20:44.812536 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7ff35-38f4-49d2-8804-6421ee09aa35 admin admin] PATH_INFO: `/v3/projects/21548439ec194a86bd7691f16ddd716a` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:20:44.813762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b7ff35-38f4-49d2-8804-6421ee09aa35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21548439ec194a86bd7691f16ddd716a)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:20:44.815702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b7ff35-38f4-49d2-8804-6421ee09aa35 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:20:44.861970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f5b7ff35-38f4-49d2-8804-6421ee09aa35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21548439ec194a86bd7691f16ddd716a', 'request_id': 'req-f5b7ff35-38f4-49d2-8804-6421ee09aa35'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:44.867958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f5b7ff35-38f4-49d2-8804-6421ee09aa35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21548439ec194a86bd7691f16ddd716a', 'request_id': 'req-f5b7ff35-38f4-49d2-8804-6421ee09aa35'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:44.875502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f5b7ff35-38f4-49d2-8804-6421ee09aa35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21548439ec194a86bd7691f16ddd716a', 'request_id': 'req-f5b7ff35-38f4-49d2-8804-6421ee09aa35'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:44.891371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f5b7ff35-38f4-49d2-8804-6421ee09aa35 admin admin] The token cache is being invalidate because project 21548439ec194a86bd7691f16ddd716a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:20:44.891702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f5b7ff35-38f4-49d2-8804-6421ee09aa35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:44.892359 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f5b7ff35-38f4-49d2-8804-6421ee09aa35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:20:44.893921 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1223/2431] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:20:44 2022] DELETE /identity/v3/projects/21548439ec194a86bd7691f16ddd716a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:20:54.543710 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=28,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:20:54.888173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=44,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:20:57.125296 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51803b-ab6f-40b4-b8e2-2d9ffc71ef7a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:20:57.126695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ac51803b-ab6f-40b4-b8e2-2d9ffc71ef7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:57.183666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51803b-ab6f-40b4-b8e2-2d9ffc71ef7a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:20:57.231393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802f6ce-85cc-45ef-87a6-66bee1390a79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:20:57.232048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802f6ce-85cc-45ef-87a6-66bee1390a79 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:20:57.232325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802f6ce-85cc-45ef-87a6-66bee1390a79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:20:57.232596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1802f6ce-85cc-45ef-87a6-66bee1390a79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:20:57.234411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1802f6ce-85cc-45ef-87a6-66bee1390a79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:20:57.300496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802f6ce-85cc-45ef-87a6-66bee1390a79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:20:57.301877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1802f6ce-85cc-45ef-87a6-66bee1390a79 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:20:57.311818 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1209/2432] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:20:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:00.388732 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7ff35-38f4-49d2-8804-6421ee09aa35 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:00.391265 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f5b7ff35-38f4-49d2-8804-6421ee09aa35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:00.419965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7ff35-38f4-49d2-8804-6421ee09aa35 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:00.425376 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a6059e-ade9-403e-a33f-d3e7d2fa9399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:00.426599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6059e-ade9-403e-a33f-d3e7d2fa9399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:00.426599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6059e-ade9-403e-a33f-d3e7d2fa9399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:00.427087 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6059e-ade9-403e-a33f-d3e7d2fa9399 admin admin] PATH_INFO: `/v3/users/01a9eebc1c8540daa31109cfc86d5ed1` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:00.428404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a6059e-ade9-403e-a33f-d3e7d2fa9399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01a9eebc1c8540daa31109cfc86d5ed1)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:00.430800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a6059e-ade9-403e-a33f-d3e7d2fa9399 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:00.489581 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f3a6059e-ade9-403e-a33f-d3e7d2fa9399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01a9eebc1c8540daa31109cfc86d5ed1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:00.490647 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f3a6059e-ade9-403e-a33f-d3e7d2fa9399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01a9eebc1c8540daa31109cfc86d5ed1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:00.505972 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f3a6059e-ade9-403e-a33f-d3e7d2fa9399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01a9eebc1c8540daa31109cfc86d5ed1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:00.514308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f3a6059e-ade9-403e-a33f-d3e7d2fa9399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01a9eebc1c8540daa31109cfc86d5ed1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:00.523037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f3a6059e-ade9-403e-a33f-d3e7d2fa9399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01a9eebc1c8540daa31109cfc86d5ed1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:00.529551 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1224/2433] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:00 2022] DELETE /identity/v3/users/01a9eebc1c8540daa31109cfc86d5ed1 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:00.720525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802f6ce-85cc-45ef-87a6-66bee1390a79 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:00.731954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1802f6ce-85cc-45ef-87a6-66bee1390a79 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:00.804271 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9293fb49-35dc-4ca7-a86f-54df31835baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:00.805270 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293fb49-35dc-4ca7-a86f-54df31835baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:00.805825 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293fb49-35dc-4ca7-a86f-54df31835baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:00.806265 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9293fb49-35dc-4ca7-a86f-54df31835baf admin admin] PATH_INFO: `/v3/projects/a4cde0820dc5474eb7091fb3a8de4cd0` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:00.807536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293fb49-35dc-4ca7-a86f-54df31835baf admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4cde0820dc5474eb7091fb3a8de4cd0)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:00.810070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9293fb49-35dc-4ca7-a86f-54df31835baf admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:00.819321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:00.861422 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9293fb49-35dc-4ca7-a86f-54df31835baf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4cde0820dc5474eb7091fb3a8de4cd0', 'request_id': 'req-9293fb49-35dc-4ca7-a86f-54df31835baf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:00.867910 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9293fb49-35dc-4ca7-a86f-54df31835baf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4cde0820dc5474eb7091fb3a8de4cd0', 'request_id': 'req-9293fb49-35dc-4ca7-a86f-54df31835baf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:00.875011 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9293fb49-35dc-4ca7-a86f-54df31835baf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4cde0820dc5474eb7091fb3a8de4cd0', 'request_id': 'req-9293fb49-35dc-4ca7-a86f-54df31835baf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:00.885115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9293fb49-35dc-4ca7-a86f-54df31835baf admin admin] The token cache is being invalidate because project a4cde0820dc5474eb7091fb3a8de4cd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:00.885394 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9293fb49-35dc-4ca7-a86f-54df31835baf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:00.886000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9293fb49-35dc-4ca7-a86f-54df31835baf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:00.887566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1210/2434] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:00 2022] DELETE /identity/v3/projects/a4cde0820dc5474eb7091fb3a8de4cd0 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:00.892503 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00f02d-ac27-48d4-a95d-97f5b9a7a53c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:00.892794 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00f02d-ac27-48d4-a95d-97f5b9a7a53c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:00.893116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00f02d-ac27-48d4-a95d-97f5b9a7a53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:00.913172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3a00f02d-ac27-48d4-a95d-97f5b9a7a53c None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:00.914388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3a00f02d-ac27-48d4-a95d-97f5b9a7a53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:00.977457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1225/2435] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:00.985914 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c968d-c67a-4478-9e18-f64884dfaee6 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:00.986198 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c968d-c67a-4478-9e18-f64884dfaee6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:00.986470 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c968d-c67a-4478-9e18-f64884dfaee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:01.004790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-1e7c968d-c67a-4478-9e18-f64884dfaee6 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:01.006008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1e7c968d-c67a-4478-9e18-f64884dfaee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:01.013895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1211/2436] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:01.018115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00f02d-ac27-48d4-a95d-97f5b9a7a53c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:01.028474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00f02d-ac27-48d4-a95d-97f5b9a7a53c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:01.033610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e0309-360a-4011-bafb-62db2a451df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:01.034589 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e0309-360a-4011-bafb-62db2a451df3 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:01.035040 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e0309-360a-4011-bafb-62db2a451df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:01.035450 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118e0309-360a-4011-bafb-62db2a451df3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:01.036477 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e0309-360a-4011-bafb-62db2a451df3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:21:01.036791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e0309-360a-4011-bafb-62db2a451df3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:01.038428 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118e0309-360a-4011-bafb-62db2a451df3 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:01.050973 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1226/2437] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:21:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:21:01.055350 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c968d-c67a-4478-9e18-f64884dfaee6 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:01.064387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c968d-c67a-4478-9e18-f64884dfaee6 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:01.068972 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415405e6-4b3f-47e7-8942-37fb1041e5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:01.069616 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415405e6-4b3f-47e7-8942-37fb1041e5fd admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:01.069890 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415405e6-4b3f-47e7-8942-37fb1041e5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:01.070156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415405e6-4b3f-47e7-8942-37fb1041e5fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:01.070842 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415405e6-4b3f-47e7-8942-37fb1041e5fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:01.072705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415405e6-4b3f-47e7-8942-37fb1041e5fd admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:01.098877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1212/2438] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:01 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:01.102982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e0309-360a-4011-bafb-62db2a451df3 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:01.113828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118e0309-360a-4011-bafb-62db2a451df3 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:01.177072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0aa1d2-2d5d-42ab-920b-2c9bf251808a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:01.177754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0aa1d2-2d5d-42ab-920b-2c9bf251808a admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:01.178038 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0aa1d2-2d5d-42ab-920b-2c9bf251808a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:01.178308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0aa1d2-2d5d-42ab-920b-2c9bf251808a admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:01.179049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0aa1d2-2d5d-42ab-920b-2c9bf251808a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:01.180975 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0aa1d2-2d5d-42ab-920b-2c9bf251808a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:01.187665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:01.203874 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1227/2439] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:01 2022] POST /identity/v3/users => generated 283 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:01.207775 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415405e6-4b3f-47e7-8942-37fb1041e5fd admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:01.218685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415405e6-4b3f-47e7-8942-37fb1041e5fd admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:01.223553 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf9277-9c71-4cd2-8e50-c8aefb4ba062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:01.224215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf9277-9c71-4cd2-8e50-c8aefb4ba062 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:01.224496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf9277-9c71-4cd2-8e50-c8aefb4ba062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:01.224790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf9277-9c71-4cd2-8e50-c8aefb4ba062 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:01.225538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf9277-9c71-4cd2-8e50-c8aefb4ba062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:01.227126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaf9277-9c71-4cd2-8e50-c8aefb4ba062 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:01.239590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1213/2440] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:01.243747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0aa1d2-2d5d-42ab-920b-2c9bf251808a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:01.254756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0aa1d2-2d5d-42ab-920b-2c9bf251808a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:01.259454 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec952c-e611-4c43-9cd1-63b1b81185b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:01.260146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec952c-e611-4c43-9cd1-63b1b81185b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:01.260454 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec952c-e611-4c43-9cd1-63b1b81185b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:01.260853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec952c-e611-4c43-9cd1-63b1b81185b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:01.261685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec952c-e611-4c43-9cd1-63b1b81185b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:01.263316 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec952c-e611-4c43-9cd1-63b1b81185b8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:01.273953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1228/2441] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:01.278673 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf9277-9c71-4cd2-8e50-c8aefb4ba062 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:01.288684 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf9277-9c71-4cd2-8e50-c8aefb4ba062 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:01.294462 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b3c36-6dfa-45b6-949a-12b0fff1ca0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:01.295476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b3c36-6dfa-45b6-949a-12b0fff1ca0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:01.296421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b3c36-6dfa-45b6-949a-12b0fff1ca0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:01.296872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13b3c36-6dfa-45b6-949a-12b0fff1ca0f admin admin] PATH_INFO: `/v3/projects/46aa882f505d43b1bbc4fd4f19ab2e2d/users/d04ee5d22a474433bcc94c63e5856755/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:01.312894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b3c36-6dfa-45b6-949a-12b0fff1ca0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=46aa882f505d43b1bbc4fd4f19ab2e2d, user_id=d04ee5d22a474433bcc94c63e5856755, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:01.315586 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13b3c36-6dfa-45b6-949a-12b0fff1ca0f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:01.328557 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1214/2442] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:01 2022] PUT /identity/v3/projects/46aa882f505d43b1bbc4fd4f19ab2e2d/users/d04ee5d22a474433bcc94c63e5856755/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:01.338937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad9725-a01b-4099-adae-02cc8547b19b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:01.339224 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad9725-a01b-4099-adae-02cc8547b19b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:01.339497 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ad9725-a01b-4099-adae-02cc8547b19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:01.366311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e0ad9725-a01b-4099-adae-02cc8547b19b None None] MFA Rules not processed for user `d04ee5d22a474433bcc94c63e5856755`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:01.367575 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e0ad9725-a01b-4099-adae-02cc8547b19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:01.431542 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1229/2443] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:01.442571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b3c36-6dfa-45b6-949a-12b0fff1ca0f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:01.444261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c13b3c36-6dfa-45b6-949a-12b0fff1ca0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:01.501094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13b3c36-6dfa-45b6-949a-12b0fff1ca0f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:01.504671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164732e-2f4c-4114-976f-7e5ba2afb69a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:01.505338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164732e-2f4c-4114-976f-7e5ba2afb69a service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:01.505622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164732e-2f4c-4114-976f-7e5ba2afb69a service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:01.505896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164732e-2f4c-4114-976f-7e5ba2afb69a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:01.514614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164732e-2f4c-4114-976f-7e5ba2afb69a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:01.516025 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164732e-2f4c-4114-976f-7e5ba2afb69a service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:01.524876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1215/2444] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:10.025196 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad9725-a01b-4099-adae-02cc8547b19b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:10.093545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ad9725-a01b-4099-adae-02cc8547b19b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:10.095791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260e68e-457b-471e-8712-7b288502268a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:10.096676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260e68e-457b-471e-8712-7b288502268a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:10.097104 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260e68e-457b-471e-8712-7b288502268a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:10.097448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260e68e-457b-471e-8712-7b288502268a admin admin] PATH_INFO: `/v3/users/d04ee5d22a474433bcc94c63e5856755` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:10.098772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260e68e-457b-471e-8712-7b288502268a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d04ee5d22a474433bcc94c63e5856755)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:10.102877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e260e68e-457b-471e-8712-7b288502268a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:10.152675 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e260e68e-457b-471e-8712-7b288502268a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd04ee5d22a474433bcc94c63e5856755'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:10.154978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e260e68e-457b-471e-8712-7b288502268a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd04ee5d22a474433bcc94c63e5856755'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:10.166758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e260e68e-457b-471e-8712-7b288502268a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd04ee5d22a474433bcc94c63e5856755'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:10.171772 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:INSERT=3,keystone:SELECT=88,keystone:DELETE=9 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:10.175833 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e260e68e-457b-471e-8712-7b288502268a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd04ee5d22a474433bcc94c63e5856755'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:10.183439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e260e68e-457b-471e-8712-7b288502268a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd04ee5d22a474433bcc94c63e5856755'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:10.191007 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1230/2445] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:10 2022] DELETE /identity/v3/users/d04ee5d22a474433bcc94c63e5856755 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:10.195908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad31344-6daf-4102-90e8-250cbd7f0af9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:10.196701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad31344-6daf-4102-90e8-250cbd7f0af9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:10.197145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad31344-6daf-4102-90e8-250cbd7f0af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:10.216269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-9ad31344-6daf-4102-90e8-250cbd7f0af9 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:10.217585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9ad31344-6daf-4102-90e8-250cbd7f0af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:10.251196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:10.282476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1216/2446] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:10.292133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260e68e-457b-471e-8712-7b288502268a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:10.293727 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e260e68e-457b-471e-8712-7b288502268a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:10.349847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260e68e-457b-471e-8712-7b288502268a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:10.353401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7ec7e-a8ff-476e-97c0-584ed73c194d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:10.354048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7ec7e-a8ff-476e-97c0-584ed73c194d service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:10.354325 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7ec7e-a8ff-476e-97c0-584ed73c194d service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:10.354596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7ec7e-a8ff-476e-97c0-584ed73c194d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:10.364016 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7ec7e-a8ff-476e-97c0-584ed73c194d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:10.365534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7ec7e-a8ff-476e-97c0-584ed73c194d service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:10.375777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1231/2447] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:10.651718 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad31344-6daf-4102-90e8-250cbd7f0af9 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:10.653678 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9ad31344-6daf-4102-90e8-250cbd7f0af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:10.665722 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad31344-6daf-4102-90e8-250cbd7f0af9 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:10.670501 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63834d14-f029-4b62-ae9b-fab298bdba1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:10.671362 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63834d14-f029-4b62-ae9b-fab298bdba1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:10.671641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63834d14-f029-4b62-ae9b-fab298bdba1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:10.671912 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63834d14-f029-4b62-ae9b-fab298bdba1e admin admin] PATH_INFO: `/v3/projects/46aa882f505d43b1bbc4fd4f19ab2e2d` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:10.673193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63834d14-f029-4b62-ae9b-fab298bdba1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=46aa882f505d43b1bbc4fd4f19ab2e2d)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:10.675139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63834d14-f029-4b62-ae9b-fab298bdba1e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:10.721230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-63834d14-f029-4b62-ae9b-fab298bdba1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46aa882f505d43b1bbc4fd4f19ab2e2d', 'request_id': 'req-63834d14-f029-4b62-ae9b-fab298bdba1e'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:10.727277 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-63834d14-f029-4b62-ae9b-fab298bdba1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46aa882f505d43b1bbc4fd4f19ab2e2d', 'request_id': 'req-63834d14-f029-4b62-ae9b-fab298bdba1e'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:10.735396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-63834d14-f029-4b62-ae9b-fab298bdba1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46aa882f505d43b1bbc4fd4f19ab2e2d', 'request_id': 'req-63834d14-f029-4b62-ae9b-fab298bdba1e'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:10.746378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-63834d14-f029-4b62-ae9b-fab298bdba1e admin admin] The token cache is being invalidate because project 46aa882f505d43b1bbc4fd4f19ab2e2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:10.746662 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-63834d14-f029-4b62-ae9b-fab298bdba1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:10.747277 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-63834d14-f029-4b62-ae9b-fab298bdba1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:10.748800 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1217/2448] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:10 2022] DELETE /identity/v3/projects/46aa882f505d43b1bbc4fd4f19ab2e2d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:10.754065 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0f403-8d6a-4286-85ff-96f121320627 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:10.754363 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0f403-8d6a-4286-85ff-96f121320627 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:10.754683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a0f403-8d6a-4286-85ff-96f121320627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:10.772730 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-84a0f403-8d6a-4286-85ff-96f121320627 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:10.773963 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-84a0f403-8d6a-4286-85ff-96f121320627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:10.831589 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1232/2449] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:10.840571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35d9fd-6cd0-413e-bd74-dfabfbb52422 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:10.840858 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35d9fd-6cd0-413e-bd74-dfabfbb52422 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:10.841188 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b35d9fd-6cd0-413e-bd74-dfabfbb52422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:10.860517 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8b35d9fd-6cd0-413e-bd74-dfabfbb52422 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:10.861794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8b35d9fd-6cd0-413e-bd74-dfabfbb52422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:10.869518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1218/2450] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:10.874388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0f403-8d6a-4286-85ff-96f121320627 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:10.883911 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a0f403-8d6a-4286-85ff-96f121320627 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:10.888495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e77fa1-2ad0-4452-aee3-381b7463e287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:10.889178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e77fa1-2ad0-4452-aee3-381b7463e287 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:10.889460 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e77fa1-2ad0-4452-aee3-381b7463e287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:10.889735 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e77fa1-2ad0-4452-aee3-381b7463e287 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:10.890461 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e77fa1-2ad0-4452-aee3-381b7463e287 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:21:10.890798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e77fa1-2ad0-4452-aee3-381b7463e287 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:10.892389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e77fa1-2ad0-4452-aee3-381b7463e287 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:10.904101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1233/2451] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:21:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:21:10.908268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35d9fd-6cd0-413e-bd74-dfabfbb52422 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:10.917951 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b35d9fd-6cd0-413e-bd74-dfabfbb52422 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:10.922653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937699a-93bb-460b-9e4c-8f4b404539bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:10.923292 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937699a-93bb-460b-9e4c-8f4b404539bf admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:10.923572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937699a-93bb-460b-9e4c-8f4b404539bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:10.923851 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937699a-93bb-460b-9e4c-8f4b404539bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:10.924533 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937699a-93bb-460b-9e4c-8f4b404539bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:10.926457 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937699a-93bb-460b-9e4c-8f4b404539bf admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:10.952395 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1219/2452] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:10 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:10.956591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e77fa1-2ad0-4452-aee3-381b7463e287 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:10.966490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e77fa1-2ad0-4452-aee3-381b7463e287 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:11.027604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d45c0-44dd-4780-a0a3-80967e75bc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:11.028310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d45c0-44dd-4780-a0a3-80967e75bc6f admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:11.028548 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d45c0-44dd-4780-a0a3-80967e75bc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:11.028819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d45c0-44dd-4780-a0a3-80967e75bc6f admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:11.029610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d45c0-44dd-4780-a0a3-80967e75bc6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:11.031583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d45c0-44dd-4780-a0a3-80967e75bc6f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:11.053095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1234/2453] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:10 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:11.056210 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937699a-93bb-460b-9e4c-8f4b404539bf admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:11.067099 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937699a-93bb-460b-9e4c-8f4b404539bf admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:11.077485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac974b7-686e-48a0-827a-188b3b977791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:11.078311 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac974b7-686e-48a0-827a-188b3b977791 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:11.078685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac974b7-686e-48a0-827a-188b3b977791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:11.079052 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac974b7-686e-48a0-827a-188b3b977791 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:11.079887 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac974b7-686e-48a0-827a-188b3b977791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:11.081609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac974b7-686e-48a0-827a-188b3b977791 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:11.093368 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1220/2454] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:11.097699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d45c0-44dd-4780-a0a3-80967e75bc6f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:11.109316 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d45c0-44dd-4780-a0a3-80967e75bc6f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:11.114177 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f5dac-c58a-4c4a-8fd6-374122f888d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:11.114782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f5dac-c58a-4c4a-8fd6-374122f888d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:11.115063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f5dac-c58a-4c4a-8fd6-374122f888d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:11.115335 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f5dac-c58a-4c4a-8fd6-374122f888d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:11.116064 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f5dac-c58a-4c4a-8fd6-374122f888d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:11.118170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f5dac-c58a-4c4a-8fd6-374122f888d5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:11.119159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:11.131258 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1235/2455] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:11.135596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac974b7-686e-48a0-827a-188b3b977791 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:11.145959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac974b7-686e-48a0-827a-188b3b977791 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:11.150598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904948cf-568e-499e-98bb-ebcd5eef3d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:11.151596 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904948cf-568e-499e-98bb-ebcd5eef3d66 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:11.152408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904948cf-568e-499e-98bb-ebcd5eef3d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:11.152679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904948cf-568e-499e-98bb-ebcd5eef3d66 admin admin] PATH_INFO: `/v3/projects/414e9de3411f47e0897ad0539e633216/users/2602a99d4e414e6cadb575c948f6fd65/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:11.168254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904948cf-568e-499e-98bb-ebcd5eef3d66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=414e9de3411f47e0897ad0539e633216, user_id=2602a99d4e414e6cadb575c948f6fd65, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:11.170894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904948cf-568e-499e-98bb-ebcd5eef3d66 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:11.183373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1221/2456] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:11 2022] PUT /identity/v3/projects/414e9de3411f47e0897ad0539e633216/users/2602a99d4e414e6cadb575c948f6fd65/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:11.193853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27ba47-51d6-4b87-b980-340689a94d24 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:11.194145 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27ba47-51d6-4b87-b980-340689a94d24 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:11.194463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27ba47-51d6-4b87-b980-340689a94d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:11.220720 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4d27ba47-51d6-4b87-b980-340689a94d24 None None] MFA Rules not processed for user `2602a99d4e414e6cadb575c948f6fd65`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:11.221944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4d27ba47-51d6-4b87-b980-340689a94d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:11.288224 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1236/2457] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:11.293560 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904948cf-568e-499e-98bb-ebcd5eef3d66 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:11.360056 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:11.365585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904948cf-568e-499e-98bb-ebcd5eef3d66 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:11.373772 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655e874-82cd-4821-b6b1-5c184cb47606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:11.373772 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655e874-82cd-4821-b6b1-5c184cb47606 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:11.373772 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655e874-82cd-4821-b6b1-5c184cb47606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:11.373772 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b655e874-82cd-4821-b6b1-5c184cb47606 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:11.375845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655e874-82cd-4821-b6b1-5c184cb47606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:11.376883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b655e874-82cd-4821-b6b1-5c184cb47606 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:11.406074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1222/2458] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:11 2022] POST /identity/v3/projects => generated 367 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:11.408978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27ba47-51d6-4b87-b980-340689a94d24 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:11.419231 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27ba47-51d6-4b87-b980-340689a94d24 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:11.482479 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb0df2-07d7-43b1-89e6-aa43bd813f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:11.483159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb0df2-07d7-43b1-89e6-aa43bd813f5c admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:11.483442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb0df2-07d7-43b1-89e6-aa43bd813f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:11.483716 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feb0df2-07d7-43b1-89e6-aa43bd813f5c admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:11.484417 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb0df2-07d7-43b1-89e6-aa43bd813f5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:11.486457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feb0df2-07d7-43b1-89e6-aa43bd813f5c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:11.509602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1237/2459] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:11 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:21:11.513969 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655e874-82cd-4821-b6b1-5c184cb47606 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:11.524693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b655e874-82cd-4821-b6b1-5c184cb47606 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:11.530035 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d5b25-5cb2-4155-84d2-79955bc6e53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:11.530695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d5b25-5cb2-4155-84d2-79955bc6e53c admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:11.530978 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d5b25-5cb2-4155-84d2-79955bc6e53c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:11.531253 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d5b25-5cb2-4155-84d2-79955bc6e53c admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:11.531986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d5b25-5cb2-4155-84d2-79955bc6e53c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:11.533607 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d5b25-5cb2-4155-84d2-79955bc6e53c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:11.544016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1223/2460] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:11.548004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb0df2-07d7-43b1-89e6-aa43bd813f5c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:11.557926 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feb0df2-07d7-43b1-89e6-aa43bd813f5c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:11.562455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104588d-1c5f-4a2c-9dcc-e5782c267817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:11.563330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104588d-1c5f-4a2c-9dcc-e5782c267817 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:11.563606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104588d-1c5f-4a2c-9dcc-e5782c267817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:11.563874 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104588d-1c5f-4a2c-9dcc-e5782c267817 admin admin] PATH_INFO: `/v3/projects/c729fce90e9a479a8eeeacf9975c350d/users/2e5b0c10b40e437394aa5f8a80a864f7/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:11.579008 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104588d-1c5f-4a2c-9dcc-e5782c267817 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c729fce90e9a479a8eeeacf9975c350d, user_id=2e5b0c10b40e437394aa5f8a80a864f7, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:11.581686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104588d-1c5f-4a2c-9dcc-e5782c267817 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:11.594537 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1238/2461] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:11 2022] PUT /identity/v3/projects/c729fce90e9a479a8eeeacf9975c350d/users/2e5b0c10b40e437394aa5f8a80a864f7/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:11.598391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d5b25-5cb2-4155-84d2-79955bc6e53c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:11.608216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d5b25-5cb2-4155-84d2-79955bc6e53c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:11.662712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec954b-d011-403f-ab4c-b4baa9afe2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:11.663361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec954b-d011-403f-ab4c-b4baa9afe2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:11.663638 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec954b-d011-403f-ab4c-b4baa9afe2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:11.663909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec954b-d011-403f-ab4c-b4baa9afe2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:11.664629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec954b-d011-403f-ab4c-b4baa9afe2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:11.666248 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec954b-d011-403f-ab4c-b4baa9afe2d4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:11.675889 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1224/2462] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:11.680057 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104588d-1c5f-4a2c-9dcc-e5782c267817 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:11.690201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104588d-1c5f-4a2c-9dcc-e5782c267817 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:11.694789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b1fc9-2ab8-40fb-908b-6fda080c8b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:11.695659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b1fc9-2ab8-40fb-908b-6fda080c8b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:11.695941 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b1fc9-2ab8-40fb-908b-6fda080c8b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:11.696215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b1fc9-2ab8-40fb-908b-6fda080c8b2a admin admin] PATH_INFO: `/v3/projects/c729fce90e9a479a8eeeacf9975c350d/users/2e5b0c10b40e437394aa5f8a80a864f7/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:11.698159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b1fc9-2ab8-40fb-908b-6fda080c8b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c729fce90e9a479a8eeeacf9975c350d, user_id=2e5b0c10b40e437394aa5f8a80a864f7, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:11.701297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70b1fc9-2ab8-40fb-908b-6fda080c8b2a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:11.711264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1239/2463] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:11 2022] PUT /identity/v3/projects/c729fce90e9a479a8eeeacf9975c350d/users/2e5b0c10b40e437394aa5f8a80a864f7/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:11.721933 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649f9d3-f9b2-403f-8ce0-a29df2a1d57d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:11.722226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649f9d3-f9b2-403f-8ce0-a29df2a1d57d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:11.722505 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649f9d3-f9b2-403f-8ce0-a29df2a1d57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:11.747973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-3649f9d3-f9b2-403f-8ce0-a29df2a1d57d None None] MFA Rules not processed for user `2e5b0c10b40e437394aa5f8a80a864f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:11.749222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3649f9d3-f9b2-403f-8ce0-a29df2a1d57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:11.819635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1225/2464] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:11 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:11.831686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b1fc9-2ab8-40fb-908b-6fda080c8b2a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:11.833439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c70b1fc9-2ab8-40fb-908b-6fda080c8b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:11.879903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:11.896880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b1fc9-2ab8-40fb-908b-6fda080c8b2a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:11.900523 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7d4ac-4e22-412c-9149-6607739c3356 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:11.901197 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7d4ac-4e22-412c-9149-6607739c3356 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:11.901481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7d4ac-4e22-412c-9149-6607739c3356 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:11.901752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba7d4ac-4e22-412c-9149-6607739c3356 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:11.910447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7d4ac-4e22-412c-9149-6607739c3356 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:11.911857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba7d4ac-4e22-412c-9149-6607739c3356 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:11.921237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1240/2465] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:11 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:11.962931 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649f9d3-f9b2-403f-8ce0-a29df2a1d57d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:11.964301 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-3649f9d3-f9b2-403f-8ce0-a29df2a1d57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:11.976020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649f9d3-f9b2-403f-8ce0-a29df2a1d57d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:11.979619 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e306f4-6485-4c3f-a90d-c2cec7649a7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:11.980356 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e306f4-6485-4c3f-a90d-c2cec7649a7a service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:11.980635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e306f4-6485-4c3f-a90d-c2cec7649a7a service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:11.980909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e306f4-6485-4c3f-a90d-c2cec7649a7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:12.027573 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:12.038190 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e306f4-6485-4c3f-a90d-c2cec7649a7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:12.039616 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e306f4-6485-4c3f-a90d-c2cec7649a7a service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:12.048482 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1226/2466] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:11 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:12.610875 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7d4ac-4e22-412c-9149-6607739c3356 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:12.612354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8ba7d4ac-4e22-412c-9149-6607739c3356 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:12.669259 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba7d4ac-4e22-412c-9149-6607739c3356 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:12.673154 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642aef38-ed2c-45ef-b0e9-367610f322c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:12.673794 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642aef38-ed2c-45ef-b0e9-367610f322c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:12.674079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642aef38-ed2c-45ef-b0e9-367610f322c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:12.674356 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642aef38-ed2c-45ef-b0e9-367610f322c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:12.676095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-642aef38-ed2c-45ef-b0e9-367610f322c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:12.737950 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642aef38-ed2c-45ef-b0e9-367610f322c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:12.739389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642aef38-ed2c-45ef-b0e9-367610f322c9 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:12.749079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1241/2467] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:15.483600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e306f4-6485-4c3f-a90d-c2cec7649a7a service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:15.494950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e306f4-6485-4c3f-a90d-c2cec7649a7a service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:15.499741 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cebe97-4b63-4503-8d02-ac8764e32c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:15.500634 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cebe97-4b63-4503-8d02-ac8764e32c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:15.500913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cebe97-4b63-4503-8d02-ac8764e32c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:15.501213 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cebe97-4b63-4503-8d02-ac8764e32c61 admin admin] PATH_INFO: `/v3/users/2602a99d4e414e6cadb575c948f6fd65` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:15.502431 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cebe97-4b63-4503-8d02-ac8764e32c61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2602a99d4e414e6cadb575c948f6fd65)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:15.504369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cebe97-4b63-4503-8d02-ac8764e32c61 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:15.562919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-40cebe97-4b63-4503-8d02-ac8764e32c61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2602a99d4e414e6cadb575c948f6fd65'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:15.563653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-40cebe97-4b63-4503-8d02-ac8764e32c61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2602a99d4e414e6cadb575c948f6fd65'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:15.582509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-40cebe97-4b63-4503-8d02-ac8764e32c61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2602a99d4e414e6cadb575c948f6fd65'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:15.590406 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-40cebe97-4b63-4503-8d02-ac8764e32c61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2602a99d4e414e6cadb575c948f6fd65'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:15.598063 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-40cebe97-4b63-4503-8d02-ac8764e32c61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2602a99d4e414e6cadb575c948f6fd65'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:15.605136 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1227/2468] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:15 2022] DELETE /identity/v3/users/2602a99d4e414e6cadb575c948f6fd65 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:15.609997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9947d-d30c-46b9-8721-ba4c44b0ff2c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:15.610490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9947d-d30c-46b9-8721-ba4c44b0ff2c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:15.610614 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a9947d-d30c-46b9-8721-ba4c44b0ff2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:15.628282 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f0a9947d-d30c-46b9-8721-ba4c44b0ff2c None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:15.629522 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f0a9947d-d30c-46b9-8721-ba4c44b0ff2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:15.693585 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1242/2469] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:15.704327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cebe97-4b63-4503-8d02-ac8764e32c61 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:15.706148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-40cebe97-4b63-4503-8d02-ac8764e32c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:15.774129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cebe97-4b63-4503-8d02-ac8764e32c61 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:15.777858 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b76f3-1584-4d0e-90b1-062e2a1d1a84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:15.778614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b76f3-1584-4d0e-90b1-062e2a1d1a84 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:15.779092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b76f3-1584-4d0e-90b1-062e2a1d1a84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:15.779435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b76f3-1584-4d0e-90b1-062e2a1d1a84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:15.792675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b76f3-1584-4d0e-90b1-062e2a1d1a84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:15.794318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b76f3-1584-4d0e-90b1-062e2a1d1a84 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:15.805091 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1228/2470] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:16.116374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9947d-d30c-46b9-8721-ba4c44b0ff2c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:16.117954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f0a9947d-d30c-46b9-8721-ba4c44b0ff2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:16.138404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a9947d-d30c-46b9-8721-ba4c44b0ff2c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:16.143916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf837477-8632-4af0-9296-7d05d1836697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:16.145040 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf837477-8632-4af0-9296-7d05d1836697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:16.145323 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf837477-8632-4af0-9296-7d05d1836697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:16.145597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf837477-8632-4af0-9296-7d05d1836697 admin admin] PATH_INFO: `/v3/projects/414e9de3411f47e0897ad0539e633216` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:16.146818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf837477-8632-4af0-9296-7d05d1836697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=414e9de3411f47e0897ad0539e633216)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:16.148726 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf837477-8632-4af0-9296-7d05d1836697 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:16.200657 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:16.223388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-cf837477-8632-4af0-9296-7d05d1836697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '414e9de3411f47e0897ad0539e633216', 'request_id': 'req-cf837477-8632-4af0-9296-7d05d1836697'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.242033 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-cf837477-8632-4af0-9296-7d05d1836697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '414e9de3411f47e0897ad0539e633216', 'request_id': 'req-cf837477-8632-4af0-9296-7d05d1836697'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.262324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-cf837477-8632-4af0-9296-7d05d1836697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '414e9de3411f47e0897ad0539e633216', 'request_id': 'req-cf837477-8632-4af0-9296-7d05d1836697'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.284092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-cf837477-8632-4af0-9296-7d05d1836697 admin admin] The token cache is being invalidate because project 414e9de3411f47e0897ad0539e633216 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:16.286420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-cf837477-8632-4af0-9296-7d05d1836697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.287746 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-cf837477-8632-4af0-9296-7d05d1836697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.290076 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1243/2471] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:16 2022] DELETE /identity/v3/projects/414e9de3411f47e0897ad0539e633216 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:16.295011 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b76f3-1584-4d0e-90b1-062e2a1d1a84 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:16.307255 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b76f3-1584-4d0e-90b1-062e2a1d1a84 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:16.394990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c6dda-91f5-4e0e-9aae-a42d413db62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:16.396079 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c6dda-91f5-4e0e-9aae-a42d413db62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:16.396363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c6dda-91f5-4e0e-9aae-a42d413db62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:16.396638 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3c6dda-91f5-4e0e-9aae-a42d413db62d admin admin] PATH_INFO: `/v3/users/2e5b0c10b40e437394aa5f8a80a864f7` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:16.398006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c6dda-91f5-4e0e-9aae-a42d413db62d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e5b0c10b40e437394aa5f8a80a864f7)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:16.400166 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3c6dda-91f5-4e0e-9aae-a42d413db62d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:16.406692 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:16.439214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf837477-8632-4af0-9296-7d05d1836697 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:16.440651 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-cf837477-8632-4af0-9296-7d05d1836697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:16.456474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf837477-8632-4af0-9296-7d05d1836697 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:16.462401 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c1e38-17d9-46d0-a498-5ea97678be7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:16.463560 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c1e38-17d9-46d0-a498-5ea97678be7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:16.463908 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c1e38-17d9-46d0-a498-5ea97678be7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:16.464703 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c1e38-17d9-46d0-a498-5ea97678be7a admin admin] PATH_INFO: `/v3/users/bff5f2d1367545f2b4c77fe0d9f095ab` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:16.465982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c1e38-17d9-46d0-a498-5ea97678be7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bff5f2d1367545f2b4c77fe0d9f095ab)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:16.467935 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c1e38-17d9-46d0-a498-5ea97678be7a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:16.470895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a3c6dda-91f5-4e0e-9aae-a42d413db62d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e5b0c10b40e437394aa5f8a80a864f7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.471584 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a3c6dda-91f5-4e0e-9aae-a42d413db62d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e5b0c10b40e437394aa5f8a80a864f7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.484563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a3c6dda-91f5-4e0e-9aae-a42d413db62d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e5b0c10b40e437394aa5f8a80a864f7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.491653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a3c6dda-91f5-4e0e-9aae-a42d413db62d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e5b0c10b40e437394aa5f8a80a864f7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.502078 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-1a3c6dda-91f5-4e0e-9aae-a42d413db62d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e5b0c10b40e437394aa5f8a80a864f7'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.509178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1229/2472] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:16 2022] DELETE /identity/v3/users/2e5b0c10b40e437394aa5f8a80a864f7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:16.517545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d79c1e38-17d9-46d0-a498-5ea97678be7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bff5f2d1367545f2b4c77fe0d9f095ab'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.519329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d79c1e38-17d9-46d0-a498-5ea97678be7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bff5f2d1367545f2b4c77fe0d9f095ab'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.534294 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d79c1e38-17d9-46d0-a498-5ea97678be7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bff5f2d1367545f2b4c77fe0d9f095ab'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.541380 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d79c1e38-17d9-46d0-a498-5ea97678be7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bff5f2d1367545f2b4c77fe0d9f095ab'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.550329 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-d79c1e38-17d9-46d0-a498-5ea97678be7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bff5f2d1367545f2b4c77fe0d9f095ab'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.556670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1244/2473] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:16 2022] DELETE /identity/v3/users/bff5f2d1367545f2b4c77fe0d9f095ab => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:21:16.745154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c6dda-91f5-4e0e-9aae-a42d413db62d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:16.746681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1a3c6dda-91f5-4e0e-9aae-a42d413db62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:16.815028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c1e38-17d9-46d0-a498-5ea97678be7a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:16.817388 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d79c1e38-17d9-46d0-a498-5ea97678be7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:16.869012 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3c6dda-91f5-4e0e-9aae-a42d413db62d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:16.898267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c1e38-17d9-46d0-a498-5ea97678be7a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:16.906067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4636a05-aa1e-4a1d-b4dd-7b2ca3c86118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:16.907107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4636a05-aa1e-4a1d-b4dd-7b2ca3c86118 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:16.907393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4636a05-aa1e-4a1d-b4dd-7b2ca3c86118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:16.907663 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4636a05-aa1e-4a1d-b4dd-7b2ca3c86118 admin admin] PATH_INFO: `/v3/projects/c729fce90e9a479a8eeeacf9975c350d` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:16.908941 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4636a05-aa1e-4a1d-b4dd-7b2ca3c86118 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c729fce90e9a479a8eeeacf9975c350d)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:16.910902 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4636a05-aa1e-4a1d-b4dd-7b2ca3c86118 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:16.936214 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fb2add-403c-4f28-8114-0bcb68113705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:16.937106 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb2add-403c-4f28-8114-0bcb68113705 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:16.937383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb2add-403c-4f28-8114-0bcb68113705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:16.937655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fb2add-403c-4f28-8114-0bcb68113705 admin admin] PATH_INFO: `/v3/projects/5056f0a32e37430ea3ea63d9cb69bf2b` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:16.938868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb2add-403c-4f28-8114-0bcb68113705 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5056f0a32e37430ea3ea63d9cb69bf2b)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:16.940778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fb2add-403c-4f28-8114-0bcb68113705 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:16.944411 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:16.963554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f4636a05-aa1e-4a1d-b4dd-7b2ca3c86118 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c729fce90e9a479a8eeeacf9975c350d', 'request_id': 'req-f4636a05-aa1e-4a1d-b4dd-7b2ca3c86118'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.971316 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f4636a05-aa1e-4a1d-b4dd-7b2ca3c86118 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c729fce90e9a479a8eeeacf9975c350d', 'request_id': 'req-f4636a05-aa1e-4a1d-b4dd-7b2ca3c86118'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.976506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:16.978802 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f4636a05-aa1e-4a1d-b4dd-7b2ca3c86118 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c729fce90e9a479a8eeeacf9975c350d', 'request_id': 'req-f4636a05-aa1e-4a1d-b4dd-7b2ca3c86118'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.990621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b2fb2add-403c-4f28-8114-0bcb68113705 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5056f0a32e37430ea3ea63d9cb69bf2b', 'request_id': 'req-b2fb2add-403c-4f28-8114-0bcb68113705'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.991985 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f4636a05-aa1e-4a1d-b4dd-7b2ca3c86118 admin admin] The token cache is being invalidate because project c729fce90e9a479a8eeeacf9975c350d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:16.992270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f4636a05-aa1e-4a1d-b4dd-7b2ca3c86118 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.992883 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-f4636a05-aa1e-4a1d-b4dd-7b2ca3c86118 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.994494 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1245/2474] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:16 2022] DELETE /identity/v3/projects/c729fce90e9a479a8eeeacf9975c350d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:16.999154 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b29fd5-f67b-4b77-83c1-01e3439bd896 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:16.999432 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b29fd5-f67b-4b77-83c1-01e3439bd896 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:16.999564 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b2fb2add-403c-4f28-8114-0bcb68113705 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5056f0a32e37430ea3ea63d9cb69bf2b', 'request_id': 'req-b2fb2add-403c-4f28-8114-0bcb68113705'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:16.999748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b29fd5-f67b-4b77-83c1-01e3439bd896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.007954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b2fb2add-403c-4f28-8114-0bcb68113705 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5056f0a32e37430ea3ea63d9cb69bf2b', 'request_id': 'req-b2fb2add-403c-4f28-8114-0bcb68113705'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:17.019124 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b2fb2add-403c-4f28-8114-0bcb68113705 admin admin] The token cache is being invalidate because project 5056f0a32e37430ea3ea63d9cb69bf2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:17.019410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b2fb2add-403c-4f28-8114-0bcb68113705 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:17.019848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-34b29fd5-f67b-4b77-83c1-01e3439bd896 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:17.020081 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b2fb2add-403c-4f28-8114-0bcb68113705 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:17.021098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-34b29fd5-f67b-4b77-83c1-01e3439bd896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:17.021645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1230/2475] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:16 2022] DELETE /identity/v3/projects/5056f0a32e37430ea3ea63d9cb69bf2b => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:17.026332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7166228-7fdf-4fe6-9ceb-47d70b4fc3a2 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.026665 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7166228-7fdf-4fe6-9ceb-47d70b4fc3a2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.027462 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7166228-7fdf-4fe6-9ceb-47d70b4fc3a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.044806 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a7166228-7fdf-4fe6-9ceb-47d70b4fc3a2 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:17.046022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a7166228-7fdf-4fe6-9ceb-47d70b4fc3a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:17.082004 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1246/2476] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:17.091411 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5162166-6ae5-4b0f-8720-571c189b152b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.091734 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5162166-6ae5-4b0f-8720-571c189b152b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.092060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5162166-6ae5-4b0f-8720-571c189b152b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.110479 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1231/2477] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:17.110886 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-b5162166-6ae5-4b0f-8720-571c189b152b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:17.112172 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-b5162166-6ae5-4b0f-8720-571c189b152b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:17.117881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59139c4c-f14e-45cf-acaf-2f4c095426b9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.118169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59139c4c-f14e-45cf-acaf-2f4c095426b9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.118441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59139c4c-f14e-45cf-acaf-2f4c095426b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.120081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1247/2478] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:17.124443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5162166-6ae5-4b0f-8720-571c189b152b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:17.133515 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5162166-6ae5-4b0f-8720-571c189b152b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:17.137598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-59139c4c-f14e-45cf-acaf-2f4c095426b9 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:17.138782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-59139c4c-f14e-45cf-acaf-2f4c095426b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:17.139001 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89189b09-a1ae-4795-886e-dcd68a994d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:17.139504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89189b09-a1ae-4795-886e-dcd68a994d33 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.139785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89189b09-a1ae-4795-886e-dcd68a994d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.140054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89189b09-a1ae-4795-886e-dcd68a994d33 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.140770 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89189b09-a1ae-4795-886e-dcd68a994d33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:21:17.141109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89189b09-a1ae-4795-886e-dcd68a994d33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:17.142678 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89189b09-a1ae-4795-886e-dcd68a994d33 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:17.146880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1232/2479] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:17.151028 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59139c4c-f14e-45cf-acaf-2f4c095426b9 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:17.154740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1248/2480] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:21:17.158767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89189b09-a1ae-4795-886e-dcd68a994d33 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:17.159986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59139c4c-f14e-45cf-acaf-2f4c095426b9 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:17.164500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4096e-46dc-449c-9edf-75ba2b537e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:17.165162 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4096e-46dc-449c-9edf-75ba2b537e81 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.165439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4096e-46dc-449c-9edf-75ba2b537e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.165710 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4096e-46dc-449c-9edf-75ba2b537e81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.166568 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4096e-46dc-449c-9edf-75ba2b537e81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:21:17.166863 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4096e-46dc-449c-9edf-75ba2b537e81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:17.168457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89189b09-a1ae-4795-886e-dcd68a994d33 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:17.168578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4096e-46dc-449c-9edf-75ba2b537e81 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:17.173067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a711531-0e5a-47c6-a70f-386d002b99ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:17.173704 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a711531-0e5a-47c6-a70f-386d002b99ca admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.173984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a711531-0e5a-47c6-a70f-386d002b99ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.174255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a711531-0e5a-47c6-a70f-386d002b99ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.174933 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a711531-0e5a-47c6-a70f-386d002b99ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:17.176792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a711531-0e5a-47c6-a70f-386d002b99ca admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:17.184404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1233/2481] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:21:17.185527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4096e-46dc-449c-9edf-75ba2b537e81 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:17.196122 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4096e-46dc-449c-9edf-75ba2b537e81 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:17.201691 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d540fc2-5b3e-403d-afe9-2ed3a23ac353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:17.202371 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d540fc2-5b3e-403d-afe9-2ed3a23ac353 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.202653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d540fc2-5b3e-403d-afe9-2ed3a23ac353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.202930 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d540fc2-5b3e-403d-afe9-2ed3a23ac353 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.203613 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d540fc2-5b3e-403d-afe9-2ed3a23ac353 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:17.204617 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1249/2482] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:17 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:21:17.205549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d540fc2-5b3e-403d-afe9-2ed3a23ac353 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:17.208339 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a711531-0e5a-47c6-a70f-386d002b99ca admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:17.232090 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1234/2483] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:17 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:21:17.239405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d540fc2-5b3e-403d-afe9-2ed3a23ac353 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:17.281798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a711531-0e5a-47c6-a70f-386d002b99ca admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:17.304432 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:17.315851 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d540fc2-5b3e-403d-afe9-2ed3a23ac353 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:17.320613 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d13a6-7066-47b6-b4e5-a42b3f833289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:17.326087 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d13a6-7066-47b6-b4e5-a42b3f833289 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.326676 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d13a6-7066-47b6-b4e5-a42b3f833289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.327052 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111d13a6-7066-47b6-b4e5-a42b3f833289 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.329423 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d13a6-7066-47b6-b4e5-a42b3f833289 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:17.332790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111d13a6-7066-47b6-b4e5-a42b3f833289 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:17.357144 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab427-2aa7-4ea1-8b44-4b1e916fee98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:17.357824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab427-2aa7-4ea1-8b44-4b1e916fee98 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.358100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab427-2aa7-4ea1-8b44-4b1e916fee98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.358379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ab427-2aa7-4ea1-8b44-4b1e916fee98 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.359101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ab427-2aa7-4ea1-8b44-4b1e916fee98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:17.360336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1235/2484] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:17 2022] POST /identity/v3/users => generated 292 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:17.361035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ab427-2aa7-4ea1-8b44-4b1e916fee98 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:17.364360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d13a6-7066-47b6-b4e5-a42b3f833289 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:17.378771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111d13a6-7066-47b6-b4e5-a42b3f833289 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:17.383467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2d40c-4df6-400a-9a00-c1c73f1dfe9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:17.384169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2d40c-4df6-400a-9a00-c1c73f1dfe9c admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.384454 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2d40c-4df6-400a-9a00-c1c73f1dfe9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.384732 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2d40c-4df6-400a-9a00-c1c73f1dfe9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.385519 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2d40c-4df6-400a-9a00-c1c73f1dfe9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:17.387685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2d40c-4df6-400a-9a00-c1c73f1dfe9c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:17.391630 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1250/2485] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:17 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:21:17.392003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:17.392589 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab427-2aa7-4ea1-8b44-4b1e916fee98 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:17.400598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1236/2486] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:21:17.404380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2d40c-4df6-400a-9a00-c1c73f1dfe9c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:17.409104 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ab427-2aa7-4ea1-8b44-4b1e916fee98 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:17.414701 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2d40c-4df6-400a-9a00-c1c73f1dfe9c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:17.416688 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887c743-32a4-4a52-95c8-792dd7154d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:17.417592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887c743-32a4-4a52-95c8-792dd7154d9c admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.418154 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887c743-32a4-4a52-95c8-792dd7154d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.418318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9887c743-32a4-4a52-95c8-792dd7154d9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.419146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887c743-32a4-4a52-95c8-792dd7154d9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:17.419408 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d37427-bc06-4bd4-9240-1296b627c208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:17.420084 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d37427-bc06-4bd4-9240-1296b627c208 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.420368 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d37427-bc06-4bd4-9240-1296b627c208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.420641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d37427-bc06-4bd4-9240-1296b627c208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.421319 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9887c743-32a4-4a52-95c8-792dd7154d9c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:17.421459 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d37427-bc06-4bd4-9240-1296b627c208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:17.422980 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d37427-bc06-4bd4-9240-1296b627c208 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:17.432322 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1251/2487] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:17.436779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887c743-32a4-4a52-95c8-792dd7154d9c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:17.442134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1237/2488] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:21:17.445033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d37427-bc06-4bd4-9240-1296b627c208 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:17.449318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9887c743-32a4-4a52-95c8-792dd7154d9c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:17.456463 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d37427-bc06-4bd4-9240-1296b627c208 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:17.458296 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df988d1e-7dbf-4eea-baaf-08b901b6d24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:17.458990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df988d1e-7dbf-4eea-baaf-08b901b6d24d admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.459269 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df988d1e-7dbf-4eea-baaf-08b901b6d24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.459539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df988d1e-7dbf-4eea-baaf-08b901b6d24d admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.460260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df988d1e-7dbf-4eea-baaf-08b901b6d24d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:17.461361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc59cb-508b-44bb-8a7d-48b9d7ffd79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:17.461875 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df988d1e-7dbf-4eea-baaf-08b901b6d24d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:17.462250 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc59cb-508b-44bb-8a7d-48b9d7ffd79a admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.462533 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc59cb-508b-44bb-8a7d-48b9d7ffd79a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.462807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc59cb-508b-44bb-8a7d-48b9d7ffd79a admin admin] PATH_INFO: `/v3/projects/a6b58555ffa54e14b2d4878f1c502792/users/f6b248b625d542c385f4da1b2a797cb4/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.473281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1252/2489] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:21:17.477222 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df988d1e-7dbf-4eea-baaf-08b901b6d24d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:17.480654 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc59cb-508b-44bb-8a7d-48b9d7ffd79a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6b58555ffa54e14b2d4878f1c502792, user_id=f6b248b625d542c385f4da1b2a797cb4, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:17.483391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc59cb-508b-44bb-8a7d-48b9d7ffd79a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:17.488629 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df988d1e-7dbf-4eea-baaf-08b901b6d24d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:17.493871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f02f0-1960-4a63-8e86-37c690caf4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:17.494844 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f02f0-1960-4a63-8e86-37c690caf4e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.495120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f02f0-1960-4a63-8e86-37c690caf4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.496018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f02f0-1960-4a63-8e86-37c690caf4e9 admin admin] PATH_INFO: `/v3/projects/5634a50fcd54440f984e73c543faf012/users/68164ec188c84818b79ae424c08def2f/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.498416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1238/2490] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:17 2022] PUT /identity/v3/projects/a6b58555ffa54e14b2d4878f1c502792/users/f6b248b625d542c385f4da1b2a797cb4/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:17.509097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0c359-6f70-4201-b379-5bd892fa6eb7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.509390 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0c359-6f70-4201-b379-5bd892fa6eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.509673 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0c359-6f70-4201-b379-5bd892fa6eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.511652 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f02f0-1960-4a63-8e86-37c690caf4e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5634a50fcd54440f984e73c543faf012, user_id=68164ec188c84818b79ae424c08def2f, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:17.514308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f02f0-1960-4a63-8e86-37c690caf4e9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:17.527781 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1253/2491] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:17 2022] PUT /identity/v3/projects/5634a50fcd54440f984e73c543faf012/users/68164ec188c84818b79ae424c08def2f/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:17.536774 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-71d0c359-6f70-4201-b379-5bd892fa6eb7 None None] MFA Rules not processed for user `f6b248b625d542c385f4da1b2a797cb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:17.538008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-71d0c359-6f70-4201-b379-5bd892fa6eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:17.538149 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed77b16-1472-4354-9fd6-489758c15fe0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.538450 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed77b16-1472-4354-9fd6-489758c15fe0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.538822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed77b16-1472-4354-9fd6-489758c15fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.566211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3ed77b16-1472-4354-9fd6-489758c15fe0 None None] MFA Rules not processed for user `68164ec188c84818b79ae424c08def2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:17.567575 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3ed77b16-1472-4354-9fd6-489758c15fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:17.604295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1239/2492] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:17.615761 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0c359-6f70-4201-b379-5bd892fa6eb7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:17.617135 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-71d0c359-6f70-4201-b379-5bd892fa6eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:17.632696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1254/2493] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:17.644014 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed77b16-1472-4354-9fd6-489758c15fe0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:17.645391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3ed77b16-1472-4354-9fd6-489758c15fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:17.691437 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0c359-6f70-4201-b379-5bd892fa6eb7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:17.696966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ad372-8ca1-4a69-a12b-8babfcd1916b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:17.698168 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ad372-8ca1-4a69-a12b-8babfcd1916b service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.698467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ad372-8ca1-4a69-a12b-8babfcd1916b service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.699198 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0ad372-8ca1-4a69-a12b-8babfcd1916b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.700856 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed77b16-1472-4354-9fd6-489758c15fe0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:17.704442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061e90da-5c97-43c1-9dce-ff76344f2953 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:17.705110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e90da-5c97-43c1-9dce-ff76344f2953 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.705386 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e90da-5c97-43c1-9dce-ff76344f2953 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.705658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061e90da-5c97-43c1-9dce-ff76344f2953 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.706979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ad372-8ca1-4a69-a12b-8babfcd1916b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:17.708567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0ad372-8ca1-4a69-a12b-8babfcd1916b service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:17.714011 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e90da-5c97-43c1-9dce-ff76344f2953 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:17.715425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061e90da-5c97-43c1-9dce-ff76344f2953 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:17.717353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1240/2494] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:17.719069 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:17.726419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1255/2495] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:21:17.747248 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ad372-8ca1-4a69-a12b-8babfcd1916b service nova] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:17.787907 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:17.811412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0ad372-8ca1-4a69-a12b-8babfcd1916b service nova] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:17.815994 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad83bf4-66c9-423f-a95c-b4c4280c23eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:17.816859 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad83bf4-66c9-423f-a95c-b4c4280c23eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.817168 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad83bf4-66c9-423f-a95c-b4c4280c23eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.817445 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad83bf4-66c9-423f-a95c-b4c4280c23eb admin admin] PATH_INFO: `/v3/users/68164ec188c84818b79ae424c08def2f` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.818645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad83bf4-66c9-423f-a95c-b4c4280c23eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=68164ec188c84818b79ae424c08def2f)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:17.820876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad83bf4-66c9-423f-a95c-b4c4280c23eb admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:17.869112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-dad83bf4-66c9-423f-a95c-b4c4280c23eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68164ec188c84818b79ae424c08def2f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:17.869894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-dad83bf4-66c9-423f-a95c-b4c4280c23eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68164ec188c84818b79ae424c08def2f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:17.883139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-dad83bf4-66c9-423f-a95c-b4c4280c23eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68164ec188c84818b79ae424c08def2f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:17.890780 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-dad83bf4-66c9-423f-a95c-b4c4280c23eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68164ec188c84818b79ae424c08def2f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:17.898417 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-dad83bf4-66c9-423f-a95c-b4c4280c23eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68164ec188c84818b79ae424c08def2f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:17.905539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1241/2496] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:17 2022] DELETE /identity/v3/users/68164ec188c84818b79ae424c08def2f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:17.910425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76806c92-f115-40fa-aec7-9ce946f5de9f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:17.910910 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76806c92-f115-40fa-aec7-9ce946f5de9f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:17.911173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76806c92-f115-40fa-aec7-9ce946f5de9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:17.929755 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-76806c92-f115-40fa-aec7-9ce946f5de9f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:17.931099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-76806c92-f115-40fa-aec7-9ce946f5de9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:18.001063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1256/2497] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:18.009375 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad83bf4-66c9-423f-a95c-b4c4280c23eb admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:18.010935 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-dad83bf4-66c9-423f-a95c-b4c4280c23eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:18.079828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad83bf4-66c9-423f-a95c-b4c4280c23eb admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:18.084041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c76418-41c7-4035-9fb1-e7b2974ba163 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:18.084915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c76418-41c7-4035-9fb1-e7b2974ba163 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:18.085262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c76418-41c7-4035-9fb1-e7b2974ba163 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:18.085540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c76418-41c7-4035-9fb1-e7b2974ba163 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:18.096083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c76418-41c7-4035-9fb1-e7b2974ba163 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:18.097794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c76418-41c7-4035-9fb1-e7b2974ba163 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:18.108604 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1242/2498] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:18.375255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76806c92-f115-40fa-aec7-9ce946f5de9f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:18.376760 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-76806c92-f115-40fa-aec7-9ce946f5de9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:18.396180 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76806c92-f115-40fa-aec7-9ce946f5de9f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:18.402291 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9d6d9f-8aae-49b9-a742-4da43196d5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:18.403222 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d6d9f-8aae-49b9-a742-4da43196d5f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:18.403504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d6d9f-8aae-49b9-a742-4da43196d5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:18.403779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9d6d9f-8aae-49b9-a742-4da43196d5f9 admin admin] PATH_INFO: `/v3/projects/5634a50fcd54440f984e73c543faf012` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:18.405006 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d6d9f-8aae-49b9-a742-4da43196d5f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5634a50fcd54440f984e73c543faf012)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:18.406898 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9d6d9f-8aae-49b9-a742-4da43196d5f9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:18.577111 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-be9d6d9f-8aae-49b9-a742-4da43196d5f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5634a50fcd54440f984e73c543faf012', 'request_id': 'req-be9d6d9f-8aae-49b9-a742-4da43196d5f9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:18.583753 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-be9d6d9f-8aae-49b9-a742-4da43196d5f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5634a50fcd54440f984e73c543faf012', 'request_id': 'req-be9d6d9f-8aae-49b9-a742-4da43196d5f9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:18.590190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-be9d6d9f-8aae-49b9-a742-4da43196d5f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5634a50fcd54440f984e73c543faf012', 'request_id': 'req-be9d6d9f-8aae-49b9-a742-4da43196d5f9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:18.602646 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-be9d6d9f-8aae-49b9-a742-4da43196d5f9 admin admin] The token cache is being invalidate because project 5634a50fcd54440f984e73c543faf012 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:18.602924 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-be9d6d9f-8aae-49b9-a742-4da43196d5f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:18.603578 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-be9d6d9f-8aae-49b9-a742-4da43196d5f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:18.605301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1257/2499] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:18 2022] DELETE /identity/v3/projects/5634a50fcd54440f984e73c543faf012 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:18.610228 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb3ff0-ebee-4a2a-ae69-28a027bcf768 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:18.611059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb3ff0-ebee-4a2a-ae69-28a027bcf768 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:18.611338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb3ff0-ebee-4a2a-ae69-28a027bcf768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:18.630572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c3eb3ff0-ebee-4a2a-ae69-28a027bcf768 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:18.631829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c3eb3ff0-ebee-4a2a-ae69-28a027bcf768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:18.681572 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:18.691406 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1243/2500] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:18.699744 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88494a11-d38b-4408-9519-3a6604a241de None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:18.700681 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88494a11-d38b-4408-9519-3a6604a241de None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:18.700983 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88494a11-d38b-4408-9519-3a6604a241de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:18.719706 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-88494a11-d38b-4408-9519-3a6604a241de None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:18.720901 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-88494a11-d38b-4408-9519-3a6604a241de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:18.728727 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1258/2501] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:18.733394 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb3ff0-ebee-4a2a-ae69-28a027bcf768 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:18.742886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb3ff0-ebee-4a2a-ae69-28a027bcf768 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:18.747410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e15967-742c-43a0-b87b-f1e0acdabc2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:18.748061 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e15967-742c-43a0-b87b-f1e0acdabc2c admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:18.748345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e15967-742c-43a0-b87b-f1e0acdabc2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:18.748622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e15967-742c-43a0-b87b-f1e0acdabc2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:18.749401 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e15967-742c-43a0-b87b-f1e0acdabc2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:21:18.749711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e15967-742c-43a0-b87b-f1e0acdabc2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:18.751300 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e15967-742c-43a0-b87b-f1e0acdabc2c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:18.762698 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1244/2502] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:21:18.766957 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88494a11-d38b-4408-9519-3a6604a241de None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:18.776273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88494a11-d38b-4408-9519-3a6604a241de None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:18.781874 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b88049-d4a2-4c76-b41f-d315342ea359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:18.782531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b88049-d4a2-4c76-b41f-d315342ea359 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:18.782810 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b88049-d4a2-4c76-b41f-d315342ea359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:18.783107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b88049-d4a2-4c76-b41f-d315342ea359 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:18.783815 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b88049-d4a2-4c76-b41f-d315342ea359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:18.785752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b88049-d4a2-4c76-b41f-d315342ea359 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:18.813275 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1259/2503] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:18 2022] POST /identity/v3/projects => generated 359 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:18.817279 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e15967-742c-43a0-b87b-f1e0acdabc2c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:18.826991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e15967-742c-43a0-b87b-f1e0acdabc2c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:18.883320 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea2d26-466f-47ea-99de-a3f5d62d3304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:18.883984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea2d26-466f-47ea-99de-a3f5d62d3304 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:18.884260 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea2d26-466f-47ea-99de-a3f5d62d3304 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:18.884531 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ea2d26-466f-47ea-99de-a3f5d62d3304 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:18.885263 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea2d26-466f-47ea-99de-a3f5d62d3304 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:18.887183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ea2d26-466f-47ea-99de-a3f5d62d3304 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:18.907996 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1245/2504] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:18 2022] POST /identity/v3/users => generated 280 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:21:18.912230 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b88049-d4a2-4c76-b41f-d315342ea359 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:18.923112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b88049-d4a2-4c76-b41f-d315342ea359 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:18.927592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e785d-ed25-481f-88db-0439037acddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:18.928243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e785d-ed25-481f-88db-0439037acddb admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:18.928520 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e785d-ed25-481f-88db-0439037acddb admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:18.928792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e785d-ed25-481f-88db-0439037acddb admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:18.929588 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e785d-ed25-481f-88db-0439037acddb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:18.931198 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e785d-ed25-481f-88db-0439037acddb admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:18.941305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1260/2505] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:18.945116 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea2d26-466f-47ea-99de-a3f5d62d3304 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:18.955065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ea2d26-466f-47ea-99de-a3f5d62d3304 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:18.959686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eab95a-ccc9-496e-b938-ef4d29981a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:18.960337 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eab95a-ccc9-496e-b938-ef4d29981a4e admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:18.960622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eab95a-ccc9-496e-b938-ef4d29981a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:18.960896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eab95a-ccc9-496e-b938-ef4d29981a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:18.961688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eab95a-ccc9-496e-b938-ef4d29981a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:18.963349 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26eab95a-ccc9-496e-b938-ef4d29981a4e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:18.973870 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1246/2506] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:18.977628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e785d-ed25-481f-88db-0439037acddb admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:18.987881 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e785d-ed25-481f-88db-0439037acddb admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:18.992396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a4b3d-6071-4f04-b3e1-9ccde95e1ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:18.993299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a4b3d-6071-4f04-b3e1-9ccde95e1ef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:18.993574 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a4b3d-6071-4f04-b3e1-9ccde95e1ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:18.993846 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a4b3d-6071-4f04-b3e1-9ccde95e1ef1 admin admin] PATH_INFO: `/v3/projects/ac80a5b9e29740878dd5bc5a67986bbf/users/a0294694956f4a76b0656010a76db496/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:19.007887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a4b3d-6071-4f04-b3e1-9ccde95e1ef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac80a5b9e29740878dd5bc5a67986bbf, user_id=a0294694956f4a76b0656010a76db496, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:19.010514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a4b3d-6071-4f04-b3e1-9ccde95e1ef1 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:19.024411 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1261/2507] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:18 2022] PUT /identity/v3/projects/ac80a5b9e29740878dd5bc5a67986bbf/users/a0294694956f4a76b0656010a76db496/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:19.035943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f3f95-ef6d-4441-99ab-2c05d63281a7 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:19.036226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f3f95-ef6d-4441-99ab-2c05d63281a7 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:19.036525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f3f95-ef6d-4441-99ab-2c05d63281a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:19.062094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-572f3f95-ef6d-4441-99ab-2c05d63281a7 None None] MFA Rules not processed for user `a0294694956f4a76b0656010a76db496`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:19.063334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-572f3f95-ef6d-4441-99ab-2c05d63281a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:19.137460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1247/2508] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:19 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:19.169685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a4b3d-6071-4f04-b3e1-9ccde95e1ef1 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:19.171348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5e6a4b3d-6071-4f04-b3e1-9ccde95e1ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:19.190805 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:19.232237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a4b3d-6071-4f04-b3e1-9ccde95e1ef1 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:19.236219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea9c7d-c212-4c14-997a-29cdcc5fcf19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:19.236893 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea9c7d-c212-4c14-997a-29cdcc5fcf19 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:19.237199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea9c7d-c212-4c14-997a-29cdcc5fcf19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:19.237469 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea9c7d-c212-4c14-997a-29cdcc5fcf19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:19.247335 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea9c7d-c212-4c14-997a-29cdcc5fcf19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:19.248965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea9c7d-c212-4c14-997a-29cdcc5fcf19 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:19.258576 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1262/2509] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:19 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:19.273389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f3f95-ef6d-4441-99ab-2c05d63281a7 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:19.340456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:19.348565 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f3f95-ef6d-4441-99ab-2c05d63281a7 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:19.353688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba877e1-98e8-41f5-a6d0-964a3f2e5de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:19.354735 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba877e1-98e8-41f5-a6d0-964a3f2e5de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:19.355018 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba877e1-98e8-41f5-a6d0-964a3f2e5de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:19.355567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba877e1-98e8-41f5-a6d0-964a3f2e5de1 admin admin] PATH_INFO: `/v3/users/a0294694956f4a76b0656010a76db496` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:19.356847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba877e1-98e8-41f5-a6d0-964a3f2e5de1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0294694956f4a76b0656010a76db496)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:19.359080 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba877e1-98e8-41f5-a6d0-964a3f2e5de1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:19.409464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fba877e1-98e8-41f5-a6d0-964a3f2e5de1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0294694956f4a76b0656010a76db496'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:19.410137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fba877e1-98e8-41f5-a6d0-964a3f2e5de1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0294694956f4a76b0656010a76db496'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:19.422145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fba877e1-98e8-41f5-a6d0-964a3f2e5de1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0294694956f4a76b0656010a76db496'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:19.429306 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fba877e1-98e8-41f5-a6d0-964a3f2e5de1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0294694956f4a76b0656010a76db496'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:19.437474 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fba877e1-98e8-41f5-a6d0-964a3f2e5de1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0294694956f4a76b0656010a76db496'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:19.443671 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1248/2510] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:19 2022] DELETE /identity/v3/users/a0294694956f4a76b0656010a76db496 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:19.448094 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b0b98-91f4-409c-a67f-73244d54288d None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:19.449035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b0b98-91f4-409c-a67f-73244d54288d None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:19.449313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b0b98-91f4-409c-a67f-73244d54288d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:19.466290 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5b2b0b98-91f4-409c-a67f-73244d54288d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:19.467484 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5b2b0b98-91f4-409c-a67f-73244d54288d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:19.524980 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1263/2511] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:19.534425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba877e1-98e8-41f5-a6d0-964a3f2e5de1 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:19.535935 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fba877e1-98e8-41f5-a6d0-964a3f2e5de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:19.596249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba877e1-98e8-41f5-a6d0-964a3f2e5de1 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:19.599808 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4045602d-bf81-4f0e-b378-46ec3c0f9db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:19.600450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045602d-bf81-4f0e-b378-46ec3c0f9db7 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:19.600728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045602d-bf81-4f0e-b378-46ec3c0f9db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:19.601034 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4045602d-bf81-4f0e-b378-46ec3c0f9db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:19.610067 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045602d-bf81-4f0e-b378-46ec3c0f9db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:19.611494 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4045602d-bf81-4f0e-b378-46ec3c0f9db7 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:19.621536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1249/2512] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:19.908426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b0b98-91f4-409c-a67f-73244d54288d None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:19.909969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5b2b0b98-91f4-409c-a67f-73244d54288d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:19.922645 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2b0b98-91f4-409c-a67f-73244d54288d None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:19.927268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7111e285-6bab-4a45-8dc7-317b31d94450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:19.928246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111e285-6bab-4a45-8dc7-317b31d94450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:19.928516 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111e285-6bab-4a45-8dc7-317b31d94450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:19.928786 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7111e285-6bab-4a45-8dc7-317b31d94450 admin admin] PATH_INFO: `/v3/projects/ac80a5b9e29740878dd5bc5a67986bbf` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:19.930041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111e285-6bab-4a45-8dc7-317b31d94450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac80a5b9e29740878dd5bc5a67986bbf)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:19.932082 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7111e285-6bab-4a45-8dc7-317b31d94450 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:19.978314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7111e285-6bab-4a45-8dc7-317b31d94450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac80a5b9e29740878dd5bc5a67986bbf', 'request_id': 'req-7111e285-6bab-4a45-8dc7-317b31d94450'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:19.985035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7111e285-6bab-4a45-8dc7-317b31d94450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac80a5b9e29740878dd5bc5a67986bbf', 'request_id': 'req-7111e285-6bab-4a45-8dc7-317b31d94450'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:19.990700 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7111e285-6bab-4a45-8dc7-317b31d94450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac80a5b9e29740878dd5bc5a67986bbf', 'request_id': 'req-7111e285-6bab-4a45-8dc7-317b31d94450'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:20.003824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7111e285-6bab-4a45-8dc7-317b31d94450 admin admin] The token cache is being invalidate because project ac80a5b9e29740878dd5bc5a67986bbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:20.004125 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7111e285-6bab-4a45-8dc7-317b31d94450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:20.004735 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7111e285-6bab-4a45-8dc7-317b31d94450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:20.006293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1264/2513] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:19 2022] DELETE /identity/v3/projects/ac80a5b9e29740878dd5bc5a67986bbf => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:20.011183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab56a7-38dd-4661-855d-16daa4c35fe5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:20.011469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab56a7-38dd-4661-855d-16daa4c35fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:20.011745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ab56a7-38dd-4661-855d-16daa4c35fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:20.029704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-b3ab56a7-38dd-4661-855d-16daa4c35fe5 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:20.030910 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b3ab56a7-38dd-4661-855d-16daa4c35fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:20.091871 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1250/2514] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:20.102033 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34d390-3cbb-4893-b064-b1be8e941b7f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:20.102408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34d390-3cbb-4893-b064-b1be8e941b7f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:20.102706 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34d390-3cbb-4893-b064-b1be8e941b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:20.123590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4d34d390-3cbb-4893-b064-b1be8e941b7f None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:20.124723 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4d34d390-3cbb-4893-b064-b1be8e941b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:20.132597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1265/2515] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:20.136857 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab56a7-38dd-4661-855d-16daa4c35fe5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:20.146831 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ab56a7-38dd-4661-855d-16daa4c35fe5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:20.151981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aab7b9-8b02-4c86-851e-9015e2aa6d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:20.152639 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aab7b9-8b02-4c86-851e-9015e2aa6d08 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:20.153025 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aab7b9-8b02-4c86-851e-9015e2aa6d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:20.153207 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aab7b9-8b02-4c86-851e-9015e2aa6d08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:20.153931 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aab7b9-8b02-4c86-851e-9015e2aa6d08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:21:20.154234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aab7b9-8b02-4c86-851e-9015e2aa6d08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:20.155922 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7aab7b9-8b02-4c86-851e-9015e2aa6d08 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:20.168456 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1251/2516] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:21:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:21:20.172689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34d390-3cbb-4893-b064-b1be8e941b7f None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:20.181710 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34d390-3cbb-4893-b064-b1be8e941b7f None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:20.186468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43766fe8-1459-4243-9e84-314f9c0a8d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:20.187112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43766fe8-1459-4243-9e84-314f9c0a8d07 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:20.187394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43766fe8-1459-4243-9e84-314f9c0a8d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:20.187671 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43766fe8-1459-4243-9e84-314f9c0a8d07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:20.188362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43766fe8-1459-4243-9e84-314f9c0a8d07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:20.190267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43766fe8-1459-4243-9e84-314f9c0a8d07 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:20.215223 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1266/2517] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:20 2022] POST /identity/v3/projects => generated 371 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:20.219276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aab7b9-8b02-4c86-851e-9015e2aa6d08 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:20.225531 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:20.230478 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aab7b9-8b02-4c86-851e-9015e2aa6d08 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:20.290797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eaa567-4586-4c9b-ade5-349722c2b3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:20.291463 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eaa567-4586-4c9b-ade5-349722c2b3b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:20.291747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eaa567-4586-4c9b-ade5-349722c2b3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:20.293261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eaa567-4586-4c9b-ade5-349722c2b3b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:20.294211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eaa567-4586-4c9b-ade5-349722c2b3b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:20.296113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eaa567-4586-4c9b-ade5-349722c2b3b7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:20.316171 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1252/2518] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:20 2022] POST /identity/v3/users => generated 286 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:20.320268 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43766fe8-1459-4243-9e84-314f9c0a8d07 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:20.330342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43766fe8-1459-4243-9e84-314f9c0a8d07 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:20.334863 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e321ea-51ec-4fb3-86da-2c456b9d019f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:20.335512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e321ea-51ec-4fb3-86da-2c456b9d019f admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:20.335793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e321ea-51ec-4fb3-86da-2c456b9d019f admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:20.336063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e321ea-51ec-4fb3-86da-2c456b9d019f admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:20.336780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e321ea-51ec-4fb3-86da-2c456b9d019f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:20.338387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e321ea-51ec-4fb3-86da-2c456b9d019f admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:20.348887 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1267/2519] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:20.352613 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eaa567-4586-4c9b-ade5-349722c2b3b7 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:20.363018 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eaa567-4586-4c9b-ade5-349722c2b3b7 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:20.367649 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda6fe6-cc68-4d06-8606-b71803013352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:20.368307 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda6fe6-cc68-4d06-8606-b71803013352 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:20.368595 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda6fe6-cc68-4d06-8606-b71803013352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:20.368870 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda6fe6-cc68-4d06-8606-b71803013352 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:20.369711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda6fe6-cc68-4d06-8606-b71803013352 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:20.371557 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda6fe6-cc68-4d06-8606-b71803013352 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:20.381440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1253/2520] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:20.385644 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e321ea-51ec-4fb3-86da-2c456b9d019f admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:20.393152 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:20.397521 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e321ea-51ec-4fb3-86da-2c456b9d019f admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:20.404638 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2a45f-6883-4a3c-a45a-e4f51a3de2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:20.405770 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2a45f-6883-4a3c-a45a-e4f51a3de2bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:20.406215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2a45f-6883-4a3c-a45a-e4f51a3de2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:20.406505 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf2a45f-6883-4a3c-a45a-e4f51a3de2bc admin admin] PATH_INFO: `/v3/projects/c2bdfb3a940b4fa3b58a781d27c51563/users/166218e825c64bd68b1494297babcb7a/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:20.421990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf2a45f-6883-4a3c-a45a-e4f51a3de2bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2bdfb3a940b4fa3b58a781d27c51563, user_id=166218e825c64bd68b1494297babcb7a, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:20.424587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf2a45f-6883-4a3c-a45a-e4f51a3de2bc admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:20.436864 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1268/2521] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:20 2022] PUT /identity/v3/projects/c2bdfb3a940b4fa3b58a781d27c51563/users/166218e825c64bd68b1494297babcb7a/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:20.447062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44fc6e-d58b-44e8-8dd3-435680ec3fc9 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:20.447349 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44fc6e-d58b-44e8-8dd3-435680ec3fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:20.447629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab44fc6e-d58b-44e8-8dd3-435680ec3fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:20.474944 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ab44fc6e-d58b-44e8-8dd3-435680ec3fc9 None None] MFA Rules not processed for user `166218e825c64bd68b1494297babcb7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:20.476170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ab44fc6e-d58b-44e8-8dd3-435680ec3fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:20.542284 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1254/2522] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:20.553695 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2a45f-6883-4a3c-a45a-e4f51a3de2bc admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:20.555398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bbf2a45f-6883-4a3c-a45a-e4f51a3de2bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:20.611133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf2a45f-6883-4a3c-a45a-e4f51a3de2bc admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:20.614954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17596ab8-962f-489b-8f76-97dbf63a59b3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b5660cfd344dcd8abe14c88df686e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:20.615607 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17596ab8-962f-489b-8f76-97dbf63a59b3 service glance] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:20.615888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17596ab8-962f-489b-8f76-97dbf63a59b3 service glance] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:20.616167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17596ab8-962f-489b-8f76-97dbf63a59b3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:20.624538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17596ab8-962f-489b-8f76-97dbf63a59b3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:20.625980 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17596ab8-962f-489b-8f76-97dbf63a59b3 service glance] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:20.634812 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1269/2523] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:21:23.054752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca9e73-4fe9-4b02-a067-50a8636022f5 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:23.055314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca9e73-4fe9-4b02-a067-50a8636022f5 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:23.055314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaca9e73-4fe9-4b02-a067-50a8636022f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:23.076291 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-eaca9e73-4fe9-4b02-a067-50a8636022f5 None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:23.077541 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-eaca9e73-4fe9-4b02-a067-50a8636022f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:23.145059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1255/2524] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Jul 14 10:21:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:23.189879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17596ab8-962f-489b-8f76-97dbf63a59b3 service glance] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:23.191305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-17596ab8-962f-489b-8f76-97dbf63a59b3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:23.276938 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17596ab8-962f-489b-8f76-97dbf63a59b3 service glance] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:23.281877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad180083-6a35-45b3-98c2-2b0f659ce2b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:23.282524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad180083-6a35-45b3-98c2-2b0f659ce2b8 service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:23.282800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad180083-6a35-45b3-98c2-2b0f659ce2b8 service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:23.283071 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad180083-6a35-45b3-98c2-2b0f659ce2b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:23.291029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad180083-6a35-45b3-98c2-2b0f659ce2b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:23.293310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad180083-6a35-45b3-98c2-2b0f659ce2b8 service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:23.301628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1270/2525] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:21:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:21:26.156215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca9e73-4fe9-4b02-a067-50a8636022f5 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:26.157272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-eaca9e73-4fe9-4b02-a067-50a8636022f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:26.180260 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:26.234927 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaca9e73-4fe9-4b02-a067-50a8636022f5 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:26.244314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d597c-f696-4d8b-b470-8c78ca5f5b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:26.245294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d597c-f696-4d8b-b470-8c78ca5f5b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:26.245599 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d597c-f696-4d8b-b470-8c78ca5f5b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:26.245825 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d597c-f696-4d8b-b470-8c78ca5f5b92 admin admin] PATH_INFO: `/v3/users/f6b248b625d542c385f4da1b2a797cb4` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:26.249650 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d597c-f696-4d8b-b470-8c78ca5f5b92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6b248b625d542c385f4da1b2a797cb4)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:26.251613 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d597c-f696-4d8b-b470-8c78ca5f5b92 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:26.321749 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-526d597c-f696-4d8b-b470-8c78ca5f5b92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6b248b625d542c385f4da1b2a797cb4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:26.322498 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-526d597c-f696-4d8b-b470-8c78ca5f5b92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6b248b625d542c385f4da1b2a797cb4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:26.347411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-526d597c-f696-4d8b-b470-8c78ca5f5b92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6b248b625d542c385f4da1b2a797cb4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:26.357203 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-526d597c-f696-4d8b-b470-8c78ca5f5b92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6b248b625d542c385f4da1b2a797cb4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:26.366302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-526d597c-f696-4d8b-b470-8c78ca5f5b92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6b248b625d542c385f4da1b2a797cb4'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:26.377043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1256/2526] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:26 2022] DELETE /identity/v3/users/f6b248b625d542c385f4da1b2a797cb4 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:26.381293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8bb1d-8ae1-4358-ad02-c689f2282ff9 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:26.382101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8bb1d-8ae1-4358-ad02-c689f2282ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:26.382544 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd8bb1d-8ae1-4358-ad02-c689f2282ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:26.413237 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-fbd8bb1d-8ae1-4358-ad02-c689f2282ff9 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:26.414534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fbd8bb1d-8ae1-4358-ad02-c689f2282ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:26.498766 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1271/2527] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:26.505366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d597c-f696-4d8b-b470-8c78ca5f5b92 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:26.507016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-526d597c-f696-4d8b-b470-8c78ca5f5b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:26.577679 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d597c-f696-4d8b-b470-8c78ca5f5b92 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:26.583246 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c8592-f267-4d56-8486-2b68652cae74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:26.584053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c8592-f267-4d56-8486-2b68652cae74 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:26.584434 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c8592-f267-4d56-8486-2b68652cae74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:26.584811 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c8592-f267-4d56-8486-2b68652cae74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:26.596143 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c8592-f267-4d56-8486-2b68652cae74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:26.597770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9c8592-f267-4d56-8486-2b68652cae74 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:26.608879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1257/2528] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:27.045036 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8bb1d-8ae1-4358-ad02-c689f2282ff9 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:27.047193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fbd8bb1d-8ae1-4358-ad02-c689f2282ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:27.070445 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd8bb1d-8ae1-4358-ad02-c689f2282ff9 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:27.078280 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ad170e-377c-481a-8d51-0c6ed6c1875c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:27.079360 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad170e-377c-481a-8d51-0c6ed6c1875c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:27.079709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad170e-377c-481a-8d51-0c6ed6c1875c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:27.080049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ad170e-377c-481a-8d51-0c6ed6c1875c admin admin] PATH_INFO: `/v3/projects/a6b58555ffa54e14b2d4878f1c502792` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:27.081346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad170e-377c-481a-8d51-0c6ed6c1875c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6b58555ffa54e14b2d4878f1c502792)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:27.083626 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ad170e-377c-481a-8d51-0c6ed6c1875c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:27.096190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:27.146818 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-37ad170e-377c-481a-8d51-0c6ed6c1875c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6b58555ffa54e14b2d4878f1c502792', 'request_id': 'req-37ad170e-377c-481a-8d51-0c6ed6c1875c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:27.154270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-37ad170e-377c-481a-8d51-0c6ed6c1875c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6b58555ffa54e14b2d4878f1c502792', 'request_id': 'req-37ad170e-377c-481a-8d51-0c6ed6c1875c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:27.159935 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-37ad170e-377c-481a-8d51-0c6ed6c1875c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6b58555ffa54e14b2d4878f1c502792', 'request_id': 'req-37ad170e-377c-481a-8d51-0c6ed6c1875c'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:27.183724 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-37ad170e-377c-481a-8d51-0c6ed6c1875c admin admin] The token cache is being invalidate because project a6b58555ffa54e14b2d4878f1c502792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:27.184613 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-37ad170e-377c-481a-8d51-0c6ed6c1875c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:27.189346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-37ad170e-377c-481a-8d51-0c6ed6c1875c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:27.191318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1272/2529] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:27 2022] DELETE /identity/v3/projects/a6b58555ffa54e14b2d4878f1c502792 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:27.196353 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612424d-81bb-4eb2-960b-7fb99cdac847 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:27.196848 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612424d-81bb-4eb2-960b-7fb99cdac847 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:27.196972 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1612424d-81bb-4eb2-960b-7fb99cdac847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:27.218382 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-1612424d-81bb-4eb2-960b-7fb99cdac847 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:27.219533 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1612424d-81bb-4eb2-960b-7fb99cdac847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:27.261187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:27.295075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1258/2530] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:27.302614 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c211763-5377-49b3-93e0-610e2c36f435 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:27.302964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c211763-5377-49b3-93e0-610e2c36f435 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:27.303178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c211763-5377-49b3-93e0-610e2c36f435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:27.340048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7c211763-5377-49b3-93e0-610e2c36f435 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:27.341636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7c211763-5377-49b3-93e0-610e2c36f435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:27.356260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1273/2531] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:27.360335 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612424d-81bb-4eb2-960b-7fb99cdac847 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:27.373972 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1612424d-81bb-4eb2-960b-7fb99cdac847 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:27.384467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c816d-1a7b-4835-bffd-a63a59c01e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:27.385178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c816d-1a7b-4835-bffd-a63a59c01e60 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:27.385461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c816d-1a7b-4835-bffd-a63a59c01e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:27.385737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c816d-1a7b-4835-bffd-a63a59c01e60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:27.386485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c816d-1a7b-4835-bffd-a63a59c01e60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:21:27.386792 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c816d-1a7b-4835-bffd-a63a59c01e60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:27.388407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c816d-1a7b-4835-bffd-a63a59c01e60 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:27.407101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1259/2532] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:21:27.408591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c211763-5377-49b3-93e0-610e2c36f435 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:27.421233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c211763-5377-49b3-93e0-610e2c36f435 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:27.424355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c6e8a-638f-436f-b389-08a4bc6bf1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:27.425133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c6e8a-638f-436f-b389-08a4bc6bf1c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:27.425504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c6e8a-638f-436f-b389-08a4bc6bf1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:27.426011 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c6e8a-638f-436f-b389-08a4bc6bf1c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:27.426786 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c6e8a-638f-436f-b389-08a4bc6bf1c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:27.428745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c6e8a-638f-436f-b389-08a4bc6bf1c9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:27.461024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1274/2533] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:27 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:27.462301 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c816d-1a7b-4835-bffd-a63a59c01e60 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:27.473810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c816d-1a7b-4835-bffd-a63a59c01e60 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:27.549916 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1e2ce-59a4-45ea-89ce-61dc01fe4e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:27.550878 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1e2ce-59a4-45ea-89ce-61dc01fe4e1f admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:27.551385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1e2ce-59a4-45ea-89ce-61dc01fe4e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:27.551658 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c1e2ce-59a4-45ea-89ce-61dc01fe4e1f admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:27.552350 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1e2ce-59a4-45ea-89ce-61dc01fe4e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:27.554539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c1e2ce-59a4-45ea-89ce-61dc01fe4e1f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:27.590156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1260/2534] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:27 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:27.594352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c6e8a-638f-436f-b389-08a4bc6bf1c9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:27.607322 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c6e8a-638f-436f-b389-08a4bc6bf1c9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:27.613547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0d8a8-3433-4a6a-9e5b-58e2485b4e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:27.614689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0d8a8-3433-4a6a-9e5b-58e2485b4e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:27.615246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0d8a8-3433-4a6a-9e5b-58e2485b4e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:27.615613 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe0d8a8-3433-4a6a-9e5b-58e2485b4e3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:27.616348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe0d8a8-3433-4a6a-9e5b-58e2485b4e3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:27.617970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe0d8a8-3433-4a6a-9e5b-58e2485b4e3a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:27.637199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1275/2535] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:27.639076 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1e2ce-59a4-45ea-89ce-61dc01fe4e1f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:27.649514 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c1e2ce-59a4-45ea-89ce-61dc01fe4e1f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:27.655736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d33bb-2611-49e2-aca0-16e609dfec0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:27.656405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d33bb-2611-49e2-aca0-16e609dfec0d admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:27.656682 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d33bb-2611-49e2-aca0-16e609dfec0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:27.656975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d33bb-2611-49e2-aca0-16e609dfec0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:27.657712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d33bb-2611-49e2-aca0-16e609dfec0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:27.659316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d33bb-2611-49e2-aca0-16e609dfec0d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:27.674629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1261/2536] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:27.679431 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0d8a8-3433-4a6a-9e5b-58e2485b4e3a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:27.694944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe0d8a8-3433-4a6a-9e5b-58e2485b4e3a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:27.701757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a9d61-5ecf-4346-bb77-583f37780c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:27.702658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a9d61-5ecf-4346-bb77-583f37780c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:27.702939 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a9d61-5ecf-4346-bb77-583f37780c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:27.703212 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a9d61-5ecf-4346-bb77-583f37780c0b admin admin] PATH_INFO: `/v3/projects/2e6d00592a534987ad4e2e4a5062b599/users/bbfe723698da479f99fc508137a3c9a3/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:27.719730 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a9d61-5ecf-4346-bb77-583f37780c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e6d00592a534987ad4e2e4a5062b599, user_id=bbfe723698da479f99fc508137a3c9a3, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:27.722600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7a9d61-5ecf-4346-bb77-583f37780c0b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:27.738095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1276/2537] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:27 2022] PUT /identity/v3/projects/2e6d00592a534987ad4e2e4a5062b599/users/bbfe723698da479f99fc508137a3c9a3/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:27.748410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc5004-7c50-4c54-9446-be2f2bf8755d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:27.748698 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc5004-7c50-4c54-9446-be2f2bf8755d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:27.749004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbc5004-7c50-4c54-9446-be2f2bf8755d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:27.785475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-cdbc5004-7c50-4c54-9446-be2f2bf8755d None None] MFA Rules not processed for user `bbfe723698da479f99fc508137a3c9a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:27.786760 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cdbc5004-7c50-4c54-9446-be2f2bf8755d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:27.879947 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1262/2538] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:27.892685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a9d61-5ecf-4346-bb77-583f37780c0b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:27.895408 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ac7a9d61-5ecf-4346-bb77-583f37780c0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:27.987289 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a9d61-5ecf-4346-bb77-583f37780c0b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:27.995145 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f349cd1-4e05-4442-8c33-ca5e5be1eb2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:27.995738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f349cd1-4e05-4442-8c33-ca5e5be1eb2a service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:27.996028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f349cd1-4e05-4442-8c33-ca5e5be1eb2a service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:27.996307 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f349cd1-4e05-4442-8c33-ca5e5be1eb2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:28.004778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f349cd1-4e05-4442-8c33-ca5e5be1eb2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:28.006567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f349cd1-4e05-4442-8c33-ca5e5be1eb2a service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:28.017250 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1277/2539] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:28.304724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc5004-7c50-4c54-9446-be2f2bf8755d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:28.352667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:28.380058 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbc5004-7c50-4c54-9446-be2f2bf8755d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:28.384656 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1793f-e281-4173-87a9-6b91601f880e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:28.386128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1793f-e281-4173-87a9-6b91601f880e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:28.386412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1793f-e281-4173-87a9-6b91601f880e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:28.386678 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b1793f-e281-4173-87a9-6b91601f880e admin admin] PATH_INFO: `/v3/users/bbfe723698da479f99fc508137a3c9a3` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:28.387876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1793f-e281-4173-87a9-6b91601f880e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbfe723698da479f99fc508137a3c9a3)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:28.389850 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b1793f-e281-4173-87a9-6b91601f880e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:28.455239 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-30b1793f-e281-4173-87a9-6b91601f880e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbfe723698da479f99fc508137a3c9a3'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:28.455954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-30b1793f-e281-4173-87a9-6b91601f880e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbfe723698da479f99fc508137a3c9a3'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:28.481908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-30b1793f-e281-4173-87a9-6b91601f880e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbfe723698da479f99fc508137a3c9a3'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:28.490113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-30b1793f-e281-4173-87a9-6b91601f880e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbfe723698da479f99fc508137a3c9a3'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:28.500003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-30b1793f-e281-4173-87a9-6b91601f880e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbfe723698da479f99fc508137a3c9a3'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:28.506984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1263/2540] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:28 2022] DELETE /identity/v3/users/bbfe723698da479f99fc508137a3c9a3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:28.511764 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b5bc8-c6bc-4cba-9fdd-a67ac24df6bd None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:28.512287 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b5bc8-c6bc-4cba-9fdd-a67ac24df6bd None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:28.512550 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b5bc8-c6bc-4cba-9fdd-a67ac24df6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:28.535541 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-127b5bc8-c6bc-4cba-9fdd-a67ac24df6bd None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:28.537170 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-127b5bc8-c6bc-4cba-9fdd-a67ac24df6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:28.555295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:28.618146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1278/2541] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:28.628823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1793f-e281-4173-87a9-6b91601f880e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:28.630403 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-30b1793f-e281-4173-87a9-6b91601f880e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:28.706933 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b1793f-e281-4173-87a9-6b91601f880e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:28.706933 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19d511-759d-4874-a9af-935c30d1abb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:28.708973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19d511-759d-4874-a9af-935c30d1abb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:28.708973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19d511-759d-4874-a9af-935c30d1abb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:28.708973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19d511-759d-4874-a9af-935c30d1abb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:28.720301 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19d511-759d-4874-a9af-935c30d1abb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:28.721147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19d511-759d-4874-a9af-935c30d1abb2 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:28.733446 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1264/2542] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:29.092052 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b5bc8-c6bc-4cba-9fdd-a67ac24df6bd None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:29.094880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-127b5bc8-c6bc-4cba-9fdd-a67ac24df6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:29.125107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b5bc8-c6bc-4cba-9fdd-a67ac24df6bd None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:29.131366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a09f7c4-ce71-42b6-b209-740338ebbba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:29.132312 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09f7c4-ce71-42b6-b209-740338ebbba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:29.132781 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09f7c4-ce71-42b6-b209-740338ebbba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:29.133146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a09f7c4-ce71-42b6-b209-740338ebbba4 admin admin] PATH_INFO: `/v3/projects/2e6d00592a534987ad4e2e4a5062b599` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:29.134416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09f7c4-ce71-42b6-b209-740338ebbba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e6d00592a534987ad4e2e4a5062b599)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:29.136461 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a09f7c4-ce71-42b6-b209-740338ebbba4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:29.193511 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7a09f7c4-ce71-42b6-b209-740338ebbba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e6d00592a534987ad4e2e4a5062b599', 'request_id': 'req-7a09f7c4-ce71-42b6-b209-740338ebbba4'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:29.200600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7a09f7c4-ce71-42b6-b209-740338ebbba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e6d00592a534987ad4e2e4a5062b599', 'request_id': 'req-7a09f7c4-ce71-42b6-b209-740338ebbba4'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:29.207472 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7a09f7c4-ce71-42b6-b209-740338ebbba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e6d00592a534987ad4e2e4a5062b599', 'request_id': 'req-7a09f7c4-ce71-42b6-b209-740338ebbba4'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:29.233603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7a09f7c4-ce71-42b6-b209-740338ebbba4 admin admin] The token cache is being invalidate because project 2e6d00592a534987ad4e2e4a5062b599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:29.233890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7a09f7c4-ce71-42b6-b209-740338ebbba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:29.234587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-7a09f7c4-ce71-42b6-b209-740338ebbba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:29.236463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1279/2543] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:29 2022] DELETE /identity/v3/projects/2e6d00592a534987ad4e2e4a5062b599 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:29.241536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16a4c4-92d4-4ba7-9da6-7c009e236543 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:29.242493 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16a4c4-92d4-4ba7-9da6-7c009e236543 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:29.243117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16a4c4-92d4-4ba7-9da6-7c009e236543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:29.262583 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8f16a4c4-92d4-4ba7-9da6-7c009e236543 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:29.264235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8f16a4c4-92d4-4ba7-9da6-7c009e236543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:29.324145 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:29.349289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1265/2544] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:29.352412 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0dabf-72eb-4324-8b1c-987b189affe0 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:29.352721 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0dabf-72eb-4324-8b1c-987b189affe0 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:29.353102 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb0dabf-72eb-4324-8b1c-987b189affe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:29.374633 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-9bb0dabf-72eb-4324-8b1c-987b189affe0 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:29.375959 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9bb0dabf-72eb-4324-8b1c-987b189affe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:29.384530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1280/2545] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:29.388574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16a4c4-92d4-4ba7-9da6-7c009e236543 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:29.403001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f16a4c4-92d4-4ba7-9da6-7c009e236543 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:29.407584 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012bfac-228a-4d91-92f1-543d2e0a8814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:29.408272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012bfac-228a-4d91-92f1-543d2e0a8814 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:29.408557 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012bfac-228a-4d91-92f1-543d2e0a8814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:29.408838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7012bfac-228a-4d91-92f1-543d2e0a8814 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:29.409765 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012bfac-228a-4d91-92f1-543d2e0a8814 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:21:29.410083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012bfac-228a-4d91-92f1-543d2e0a8814 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:29.411717 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7012bfac-228a-4d91-92f1-543d2e0a8814 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:29.431570 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1266/2546] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:21:29.434568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0dabf-72eb-4324-8b1c-987b189affe0 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:29.446096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb0dabf-72eb-4324-8b1c-987b189affe0 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:29.451273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670838b-e166-4d2f-abc1-908774d45ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:29.452082 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670838b-e166-4d2f-abc1-908774d45ecf admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:29.452424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670838b-e166-4d2f-abc1-908774d45ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:29.452648 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670838b-e166-4d2f-abc1-908774d45ecf admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:29.453341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670838b-e166-4d2f-abc1-908774d45ecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:29.455245 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670838b-e166-4d2f-abc1-908774d45ecf admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:29.493001 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1281/2547] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:29 2022] POST /identity/v3/projects => generated 355 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:29.495781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012bfac-228a-4d91-92f1-543d2e0a8814 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:29.512154 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7012bfac-228a-4d91-92f1-543d2e0a8814 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:29.612769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0916d-1c5a-4b5b-ac17-72138c9f975a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:29.613436 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0916d-1c5a-4b5b-ac17-72138c9f975a admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:29.613718 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0916d-1c5a-4b5b-ac17-72138c9f975a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:29.613995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0916d-1c5a-4b5b-ac17-72138c9f975a admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:29.614708 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0916d-1c5a-4b5b-ac17-72138c9f975a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:29.616635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c0916d-1c5a-4b5b-ac17-72138c9f975a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:29.644628 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1267/2548] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:29 2022] POST /identity/v3/users => generated 278 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:29.646149 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670838b-e166-4d2f-abc1-908774d45ecf admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:29.662847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670838b-e166-4d2f-abc1-908774d45ecf admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:29.668260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136bdad6-186a-4810-8760-f050c5d8f78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:29.669427 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bdad6-186a-4810-8760-f050c5d8f78d admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:29.669764 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bdad6-186a-4810-8760-f050c5d8f78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:29.669989 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bdad6-186a-4810-8760-f050c5d8f78d admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:29.670899 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136bdad6-186a-4810-8760-f050c5d8f78d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:29.672731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136bdad6-186a-4810-8760-f050c5d8f78d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:29.684355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1282/2549] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:21:29.690923 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0916d-1c5a-4b5b-ac17-72138c9f975a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:29.705412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c0916d-1c5a-4b5b-ac17-72138c9f975a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:29.711342 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22a276b-c20f-428d-84d8-364904c0a76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:29.712040 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a276b-c20f-428d-84d8-364904c0a76e admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:29.712324 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a276b-c20f-428d-84d8-364904c0a76e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:29.712601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22a276b-c20f-428d-84d8-364904c0a76e admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:29.713439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a276b-c20f-428d-84d8-364904c0a76e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:29.715604 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22a276b-c20f-428d-84d8-364904c0a76e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:29.727578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1268/2550] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:29.732034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136bdad6-186a-4810-8760-f050c5d8f78d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:29.742426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136bdad6-186a-4810-8760-f050c5d8f78d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:29.753567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf9de1-4ec8-4662-aa37-1b2770257037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:29.753567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf9de1-4ec8-4662-aa37-1b2770257037 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:29.753567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf9de1-4ec8-4662-aa37-1b2770257037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:29.753567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf9de1-4ec8-4662-aa37-1b2770257037 admin admin] PATH_INFO: `/v3/projects/33f8f2179c154f77a2dbb0e53f8b6289/users/c3d8f3d065c243199fcc89d4753f7829/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:29.776460 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf9de1-4ec8-4662-aa37-1b2770257037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33f8f2179c154f77a2dbb0e53f8b6289, user_id=c3d8f3d065c243199fcc89d4753f7829, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:29.779374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faf9de1-4ec8-4662-aa37-1b2770257037 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:29.807985 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1283/2551] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:29 2022] PUT /identity/v3/projects/33f8f2179c154f77a2dbb0e53f8b6289/users/c3d8f3d065c243199fcc89d4753f7829/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:29.810065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebeb0c-f95c-49ec-b6b1-c09215edba14 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:29.810424 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebeb0c-f95c-49ec-b6b1-c09215edba14 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:29.810703 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebeb0c-f95c-49ec-b6b1-c09215edba14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:29.848414 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-38ebeb0c-f95c-49ec-b6b1-c09215edba14 None None] MFA Rules not processed for user `c3d8f3d065c243199fcc89d4753f7829`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:29.851031 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-38ebeb0c-f95c-49ec-b6b1-c09215edba14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:29.956196 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1269/2552] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:29 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:29.961098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf9de1-4ec8-4662-aa37-1b2770257037 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:30.012134 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:30.050289 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf9de1-4ec8-4662-aa37-1b2770257037 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:30.060083 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855af27-c174-4c41-ac50-9bf8b9017146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:30.060958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855af27-c174-4c41-ac50-9bf8b9017146 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:30.061372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855af27-c174-4c41-ac50-9bf8b9017146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:30.061747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855af27-c174-4c41-ac50-9bf8b9017146 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:30.062659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855af27-c174-4c41-ac50-9bf8b9017146 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:30.064708 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855af27-c174-4c41-ac50-9bf8b9017146 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:30.105159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1284/2553] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:29 2022] POST /identity/v3/projects => generated 355 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:30.109579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebeb0c-f95c-49ec-b6b1-c09215edba14 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:30.130397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebeb0c-f95c-49ec-b6b1-c09215edba14 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:30.227551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:30.240576 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38fc67-c4e7-4078-8625-907d678796a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:30.245217 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38fc67-c4e7-4078-8625-907d678796a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:30.245616 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38fc67-c4e7-4078-8625-907d678796a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:30.245992 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de38fc67-c4e7-4078-8625-907d678796a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:30.246764 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38fc67-c4e7-4078-8625-907d678796a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:30.249639 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de38fc67-c4e7-4078-8625-907d678796a7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:30.286216 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855af27-c174-4c41-ac50-9bf8b9017146 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:30.286729 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1270/2554] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:30 2022] POST /identity/v3/users => generated 284 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:30.301215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855af27-c174-4c41-ac50-9bf8b9017146 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:30.306091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be5df3-b635-4a92-889a-af1645202689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:30.307073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be5df3-b635-4a92-889a-af1645202689 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:30.307454 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be5df3-b635-4a92-889a-af1645202689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:30.307788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be5df3-b635-4a92-889a-af1645202689 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:30.308660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be5df3-b635-4a92-889a-af1645202689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:30.310405 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be5df3-b635-4a92-889a-af1645202689 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:30.330101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1285/2555] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:30.331289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38fc67-c4e7-4078-8625-907d678796a7 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:30.346389 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de38fc67-c4e7-4078-8625-907d678796a7 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:30.357869 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b21dc-a21d-4d49-a2b0-0c74c41147e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:30.358888 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b21dc-a21d-4d49-a2b0-0c74c41147e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:30.359232 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b21dc-a21d-4d49-a2b0-0c74c41147e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:30.359574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774b21dc-a21d-4d49-a2b0-0c74c41147e9 admin admin] PATH_INFO: `/v3/projects/eaaf1448908e48fdacbd1b284571466b/users/9ab2b71222cb4a85958ee2d87fde2236/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:30.378054 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774b21dc-a21d-4d49-a2b0-0c74c41147e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaaf1448908e48fdacbd1b284571466b, user_id=9ab2b71222cb4a85958ee2d87fde2236, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:30.380989 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774b21dc-a21d-4d49-a2b0-0c74c41147e9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:30.405504 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1271/2556] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:30 2022] PUT /identity/v3/projects/eaaf1448908e48fdacbd1b284571466b/users/9ab2b71222cb4a85958ee2d87fde2236/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:30.408596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be5df3-b635-4a92-889a-af1645202689 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:30.424271 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be5df3-b635-4a92-889a-af1645202689 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:30.496466 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec35680-7d93-40fe-85f5-adde8aba4a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:30.497193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec35680-7d93-40fe-85f5-adde8aba4a3a admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:30.497468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec35680-7d93-40fe-85f5-adde8aba4a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:30.497742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec35680-7d93-40fe-85f5-adde8aba4a3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:30.498484 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec35680-7d93-40fe-85f5-adde8aba4a3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:30.500096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec35680-7d93-40fe-85f5-adde8aba4a3a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:30.513120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1286/2557] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:30 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:30.515873 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b21dc-a21d-4d49-a2b0-0c74c41147e9 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:30.533783 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774b21dc-a21d-4d49-a2b0-0c74c41147e9 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:30.539490 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58d22f-1cb2-4db4-b37d-b0e9fac6b748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:30.540856 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58d22f-1cb2-4db4-b37d-b0e9fac6b748 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:30.541473 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58d22f-1cb2-4db4-b37d-b0e9fac6b748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:30.543178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58d22f-1cb2-4db4-b37d-b0e9fac6b748 admin admin] PATH_INFO: `/v3/projects/eaaf1448908e48fdacbd1b284571466b/users/9ab2b71222cb4a85958ee2d87fde2236/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:30.548242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58d22f-1cb2-4db4-b37d-b0e9fac6b748 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaaf1448908e48fdacbd1b284571466b, user_id=9ab2b71222cb4a85958ee2d87fde2236, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:30.551267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a58d22f-1cb2-4db4-b37d-b0e9fac6b748 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:30.563649 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1272/2558] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:30 2022] PUT /identity/v3/projects/eaaf1448908e48fdacbd1b284571466b/users/9ab2b71222cb4a85958ee2d87fde2236/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:30.580442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa584565-5f89-4060-96b2-922c0ef37210 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:30.580825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa584565-5f89-4060-96b2-922c0ef37210 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:30.581415 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa584565-5f89-4060-96b2-922c0ef37210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:30.640281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-fa584565-5f89-4060-96b2-922c0ef37210 None None] MFA Rules not processed for user `9ab2b71222cb4a85958ee2d87fde2236`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:30.641665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fa584565-5f89-4060-96b2-922c0ef37210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:30.763449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1287/2559] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:30 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:30.778927 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58d22f-1cb2-4db4-b37d-b0e9fac6b748 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:30.780854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7a58d22f-1cb2-4db4-b37d-b0e9fac6b748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:30.872300 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58d22f-1cb2-4db4-b37d-b0e9fac6b748 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:30.877961 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3586d531-7bb6-4742-9960-b5dfa7dd35a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:30.878774 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586d531-7bb6-4742-9960-b5dfa7dd35a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:30.879263 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586d531-7bb6-4742-9960-b5dfa7dd35a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:30.879745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3586d531-7bb6-4742-9960-b5dfa7dd35a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:30.891280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586d531-7bb6-4742-9960-b5dfa7dd35a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:30.891964 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3586d531-7bb6-4742-9960-b5dfa7dd35a7 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:30.903789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1273/2560] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:30 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:30.987792 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa584565-5f89-4060-96b2-922c0ef37210 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:31.000847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa584565-5f89-4060-96b2-922c0ef37210 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:31.005260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5dc964-6fbf-46f9-be69-3ed6663e880d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:31.006116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dc964-6fbf-46f9-be69-3ed6663e880d service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:31.006397 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dc964-6fbf-46f9-be69-3ed6663e880d service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:31.006669 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5dc964-6fbf-46f9-be69-3ed6663e880d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:31.044163 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:31.075822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dc964-6fbf-46f9-be69-3ed6663e880d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:31.077304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5dc964-6fbf-46f9-be69-3ed6663e880d service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:31.090799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1288/2561] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:30 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:21:36.227270 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73582cca-e76b-42a1-a25e-3719109ae723 None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:36.227792 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73582cca-e76b-42a1-a25e-3719109ae723 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:36.227877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73582cca-e76b-42a1-a25e-3719109ae723 None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:36.229251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1274/2562] 10.222.0.61 () {54 vars in 921 bytes} [Thu Jul 14 10:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:21:36.235534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc15377-1739-412d-9abb-0a9a70b5ead3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:36.236000 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc15377-1739-412d-9abb-0a9a70b5ead3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:36.236211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc15377-1739-412d-9abb-0a9a70b5ead3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:36.254781 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8bc15377-1739-412d-9abb-0a9a70b5ead3 None None] MFA Rules not processed for user `e9f4efa3e2f343c2ac858a277601a7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:36.256015 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8bc15377-1739-412d-9abb-0a9a70b5ead3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:36.325710 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1289/2563] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Jul 14 10:21:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:36.336764 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73582cca-e76b-42a1-a25e-3719109ae723 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:36.338635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-73582cca-e76b-42a1-a25e-3719109ae723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:36.402829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73582cca-e76b-42a1-a25e-3719109ae723 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:36.406437 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddb9442-f50b-4ed6-9ce0-2f574bbfd802 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b5660cfd344dcd8abe14c88df686e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:36.407116 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb9442-f50b-4ed6-9ce0-2f574bbfd802 service glance] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:36.407400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb9442-f50b-4ed6-9ce0-2f574bbfd802 service glance] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:36.407676 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddb9442-f50b-4ed6-9ce0-2f574bbfd802 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:36.418547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb9442-f50b-4ed6-9ce0-2f574bbfd802 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:36.420097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddb9442-f50b-4ed6-9ce0-2f574bbfd802 service glance] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:36.431362 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1275/2564] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:36.503371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cc5e6-64a8-4ea2-a86d-baba5d548ead None None] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:36.503655 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cc5e6-64a8-4ea2-a86d-baba5d548ead None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:36.504071 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6cc5e6-64a8-4ea2-a86d-baba5d548ead None None] PATH_INFO: `/` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:36.505493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1290/2565] 10.222.0.61 () {54 vars in 921 bytes} [Thu Jul 14 10:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 14 10:21:36.511872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51908621-4c8d-4d79-9d32-913ea897c4fc None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:36.512436 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51908621-4c8d-4d79-9d32-913ea897c4fc None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:36.512763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51908621-4c8d-4d79-9d32-913ea897c4fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:36.533122 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-51908621-4c8d-4d79-9d32-913ea897c4fc None None] MFA Rules not processed for user `cd6d19c4406b4b228065b98a9a074b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:36.534313 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-51908621-4c8d-4d79-9d32-913ea897c4fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:36.578060 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:36.585823 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1276/2566] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Jul 14 10:21:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:36.601352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cc5e6-64a8-4ea2-a86d-baba5d548ead None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:36.603077 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8f6cc5e6-64a8-4ea2-a86d-baba5d548ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:36.615989 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6cc5e6-64a8-4ea2-a86d-baba5d548ead None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:36.619470 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2343f19-b540-441e-b8d4-7a269db351c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:36.620098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2343f19-b540-441e-b8d4-7a269db351c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:36.620876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2343f19-b540-441e-b8d4-7a269db351c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:36.621200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2343f19-b540-441e-b8d4-7a269db351c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:36.628391 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2343f19-b540-441e-b8d4-7a269db351c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:36.629853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2343f19-b540-441e-b8d4-7a269db351c0 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:36.638261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1291/2567] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:36.719868 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2827c5b-1a3f-4013-b0e7-5d3c8ad94d6a None None] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:36.720160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2827c5b-1a3f-4013-b0e7-5d3c8ad94d6a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:36.720467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2827c5b-1a3f-4013-b0e7-5d3c8ad94d6a None None] PATH_INFO: `/` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:36.721971 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1277/2568] 10.222.0.61 () {54 vars in 921 bytes} [Thu Jul 14 10:21:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 14 10:21:36.726946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85e725-6d60-42fd-8489-780b94bb5230 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:36.727235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85e725-6d60-42fd-8489-780b94bb5230 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:36.727510 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d85e725-6d60-42fd-8489-780b94bb5230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:36.746335 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0d85e725-6d60-42fd-8489-780b94bb5230 None None] MFA Rules not processed for user `e9f4efa3e2f343c2ac858a277601a7df`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:36.747634 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0d85e725-6d60-42fd-8489-780b94bb5230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:36.756876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1292/2569] 10.222.0.61 () {58 vars in 1018 bytes} [Thu Jul 14 10:21:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:36.768605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2827c5b-1a3f-4013-b0e7-5d3c8ad94d6a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:36.770316 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-d2827c5b-1a3f-4013-b0e7-5d3c8ad94d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:36.785998 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2827c5b-1a3f-4013-b0e7-5d3c8ad94d6a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:36.790293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7505f23-6386-4924-9527-58d089ae0d6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:36.790942 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7505f23-6386-4924-9527-58d089ae0d6d service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:36.791311 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7505f23-6386-4924-9527-58d089ae0d6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:36.791583 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7505f23-6386-4924-9527-58d089ae0d6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:36.802012 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7505f23-6386-4924-9527-58d089ae0d6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:36.803476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7505f23-6386-4924-9527-58d089ae0d6d service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:36.815040 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1278/2570] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:39.040442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e888093-ca7d-4849-ab3b-cf5e1c3e966e None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:39.040859 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e888093-ca7d-4849-ab3b-cf5e1c3e966e None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:39.041110 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e888093-ca7d-4849-ab3b-cf5e1c3e966e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:39.059238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3e888093-ca7d-4849-ab3b-cf5e1c3e966e None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:39.060433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3e888093-ca7d-4849-ab3b-cf5e1c3e966e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:39.118477 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1293/2571] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Jul 14 10:21:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:39.130313 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7505f23-6386-4924-9527-58d089ae0d6d service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:39.131695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c7505f23-6386-4924-9527-58d089ae0d6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:39.204102 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7505f23-6386-4924-9527-58d089ae0d6d service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:39.209192 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5f298-974e-4f0e-9052-7ad9977561f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:39.209843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5f298-974e-4f0e-9052-7ad9977561f3 service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:39.210204 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5f298-974e-4f0e-9052-7ad9977561f3 service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:39.210590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e5f298-974e-4f0e-9052-7ad9977561f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:39.220304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5f298-974e-4f0e-9052-7ad9977561f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:39.222158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e5f298-974e-4f0e-9052-7ad9977561f3 service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:39.230401 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1279/2572] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:21:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:21:48.487880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e888093-ca7d-4849-ab3b-cf5e1c3e966e None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:48.562821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5f298-974e-4f0e-9052-7ad9977561f3 service swift] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:48.564698 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-86e5f298-974e-4f0e-9052-7ad9977561f3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:48.599872 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:48.671022 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e888093-ca7d-4849-ab3b-cf5e1c3e966e None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:48.678979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2185449-d36e-46de-8221-e8c443f45125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:48.684638 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2185449-d36e-46de-8221-e8c443f45125 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:48.685208 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2185449-d36e-46de-8221-e8c443f45125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:48.686501 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2185449-d36e-46de-8221-e8c443f45125 admin admin] PATH_INFO: `/v3/users/c3d8f3d065c243199fcc89d4753f7829` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:48.689504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2185449-d36e-46de-8221-e8c443f45125 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3d8f3d065c243199fcc89d4753f7829)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:48.691417 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2185449-d36e-46de-8221-e8c443f45125 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:48.730635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e5f298-974e-4f0e-9052-7ad9977561f3 service swift] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:48.735399 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b18bd3-3e7b-45c5-bd0a-e3a7be960722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:48.736282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b18bd3-3e7b-45c5-bd0a-e3a7be960722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:48.736563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b18bd3-3e7b-45c5-bd0a-e3a7be960722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:48.736833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b18bd3-3e7b-45c5-bd0a-e3a7be960722 admin admin] PATH_INFO: `/v3/users/166218e825c64bd68b1494297babcb7a` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:48.738089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b18bd3-3e7b-45c5-bd0a-e3a7be960722 admin admin] RBAC: Authorizing `identity:delete_user(user_id=166218e825c64bd68b1494297babcb7a)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:48.740096 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b18bd3-3e7b-45c5-bd0a-e3a7be960722 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:48.808020 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b2185449-d36e-46de-8221-e8c443f45125 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3d8f3d065c243199fcc89d4753f7829'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:48.809438 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b2185449-d36e-46de-8221-e8c443f45125 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3d8f3d065c243199fcc89d4753f7829'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:48.813804 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:48.831234 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-78b18bd3-3e7b-45c5-bd0a-e3a7be960722 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '166218e825c64bd68b1494297babcb7a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:48.833383 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-78b18bd3-3e7b-45c5-bd0a-e3a7be960722 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '166218e825c64bd68b1494297babcb7a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:48.839694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b2185449-d36e-46de-8221-e8c443f45125 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3d8f3d065c243199fcc89d4753f7829'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:48.856870 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b2185449-d36e-46de-8221-e8c443f45125 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3d8f3d065c243199fcc89d4753f7829'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:48.862852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-78b18bd3-3e7b-45c5-bd0a-e3a7be960722 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '166218e825c64bd68b1494297babcb7a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:48.884347 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-b2185449-d36e-46de-8221-e8c443f45125 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3d8f3d065c243199fcc89d4753f7829'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:48.886844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-78b18bd3-3e7b-45c5-bd0a-e3a7be960722 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '166218e825c64bd68b1494297babcb7a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:48.893439 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1294/2573] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:48 2022] DELETE /identity/v3/users/c3d8f3d065c243199fcc89d4753f7829 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:48.902095 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ee6e1-4bf4-4da7-b2c9-795feee9a0c4 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:48.902409 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ee6e1-4bf4-4da7-b2c9-795feee9a0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:48.902684 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ee6e1-4bf4-4da7-b2c9-795feee9a0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:48.910696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-78b18bd3-3e7b-45c5-bd0a-e3a7be960722 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '166218e825c64bd68b1494297babcb7a'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:48.921050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1280/2574] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:48 2022] DELETE /identity/v3/users/166218e825c64bd68b1494297babcb7a => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:21:48.922496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af588ed7-6d48-42a2-af1b-49b5e64e490d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:48.922733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af588ed7-6d48-42a2-af1b-49b5e64e490d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:48.923051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af588ed7-6d48-42a2-af1b-49b5e64e490d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:48.941715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-897ee6e1-4bf4-4da7-b2c9-795feee9a0c4 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:48.942669 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-af588ed7-6d48-42a2-af1b-49b5e64e490d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:48.943085 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-897ee6e1-4bf4-4da7-b2c9-795feee9a0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:48.943850 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-af588ed7-6d48-42a2-af1b-49b5e64e490d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:49.004217 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1281/2575] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:49.022015 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1295/2576] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:49.026100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af588ed7-6d48-42a2-af1b-49b5e64e490d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:49.028167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-af588ed7-6d48-42a2-af1b-49b5e64e490d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:49.029160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ee6e1-4bf4-4da7-b2c9-795feee9a0c4 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:49.030601 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-897ee6e1-4bf4-4da7-b2c9-795feee9a0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:49.054857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:49.115394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ee6e1-4bf4-4da7-b2c9-795feee9a0c4 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:49.119163 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93150326-a2ef-40bf-b06b-a75a04de9eb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:49.120382 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93150326-a2ef-40bf-b06b-a75a04de9eb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:49.120661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93150326-a2ef-40bf-b06b-a75a04de9eb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:49.120951 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93150326-a2ef-40bf-b06b-a75a04de9eb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:49.126862 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af588ed7-6d48-42a2-af1b-49b5e64e490d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:49.131205 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93150326-a2ef-40bf-b06b-a75a04de9eb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:49.131805 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca6ff4-a486-45c1-a80a-754e47f75057 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:49.132659 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93150326-a2ef-40bf-b06b-a75a04de9eb9 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:49.132797 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca6ff4-a486-45c1-a80a-754e47f75057 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:49.133270 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca6ff4-a486-45c1-a80a-754e47f75057 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:49.133641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca6ff4-a486-45c1-a80a-754e47f75057 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:49.143200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1296/2577] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:21:49.146345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca6ff4-a486-45c1-a80a-754e47f75057 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:49.147904 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dca6ff4-a486-45c1-a80a-754e47f75057 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:49.158059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1282/2578] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:49.525181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93150326-a2ef-40bf-b06b-a75a04de9eb9 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:49.527830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-93150326-a2ef-40bf-b06b-a75a04de9eb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:49.539678 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca6ff4-a486-45c1-a80a-754e47f75057 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:49.541131 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5dca6ff4-a486-45c1-a80a-754e47f75057 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:49.554598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93150326-a2ef-40bf-b06b-a75a04de9eb9 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:49.557654 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca6ff4-a486-45c1-a80a-754e47f75057 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:49.560205 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae021b-0690-4312-8e0f-4df1c355f294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:49.561171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae021b-0690-4312-8e0f-4df1c355f294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:49.561481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae021b-0690-4312-8e0f-4df1c355f294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:49.561706 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ae021b-0690-4312-8e0f-4df1c355f294 admin admin] PATH_INFO: `/v3/projects/33f8f2179c154f77a2dbb0e53f8b6289` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:49.562999 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae021b-0690-4312-8e0f-4df1c355f294 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33f8f2179c154f77a2dbb0e53f8b6289)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:49.564958 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e6622-02a6-43e0-9f3f-1399172f4e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:49.565509 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ae021b-0690-4312-8e0f-4df1c355f294 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:49.566159 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6622-02a6-43e0-9f3f-1399172f4e1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:49.566448 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6622-02a6-43e0-9f3f-1399172f4e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:49.566726 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e6622-02a6-43e0-9f3f-1399172f4e1d admin admin] PATH_INFO: `/v3/projects/c2bdfb3a940b4fa3b58a781d27c51563` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:49.568186 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6622-02a6-43e0-9f3f-1399172f4e1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2bdfb3a940b4fa3b58a781d27c51563)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:49.570604 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e6622-02a6-43e0-9f3f-1399172f4e1d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:49.610494 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:49.615183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91ae021b-0690-4312-8e0f-4df1c355f294 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33f8f2179c154f77a2dbb0e53f8b6289', 'request_id': 'req-91ae021b-0690-4312-8e0f-4df1c355f294'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:49.621939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b86e6622-02a6-43e0-9f3f-1399172f4e1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2bdfb3a940b4fa3b58a781d27c51563', 'request_id': 'req-b86e6622-02a6-43e0-9f3f-1399172f4e1d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:49.623245 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91ae021b-0690-4312-8e0f-4df1c355f294 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33f8f2179c154f77a2dbb0e53f8b6289', 'request_id': 'req-91ae021b-0690-4312-8e0f-4df1c355f294'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:49.628891 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b86e6622-02a6-43e0-9f3f-1399172f4e1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2bdfb3a940b4fa3b58a781d27c51563', 'request_id': 'req-b86e6622-02a6-43e0-9f3f-1399172f4e1d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:49.629142 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91ae021b-0690-4312-8e0f-4df1c355f294 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33f8f2179c154f77a2dbb0e53f8b6289', 'request_id': 'req-91ae021b-0690-4312-8e0f-4df1c355f294'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:49.639230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b86e6622-02a6-43e0-9f3f-1399172f4e1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2bdfb3a940b4fa3b58a781d27c51563', 'request_id': 'req-b86e6622-02a6-43e0-9f3f-1399172f4e1d'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:49.642713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91ae021b-0690-4312-8e0f-4df1c355f294 admin admin] The token cache is being invalidate because project 33f8f2179c154f77a2dbb0e53f8b6289 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:49.643001 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91ae021b-0690-4312-8e0f-4df1c355f294 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:49.643632 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91ae021b-0690-4312-8e0f-4df1c355f294 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:49.645231 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1297/2579] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:49 2022] DELETE /identity/v3/projects/33f8f2179c154f77a2dbb0e53f8b6289 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:49.648993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae021b-0690-4312-8e0f-4df1c355f294 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:49.650324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-91ae021b-0690-4312-8e0f-4df1c355f294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:49.654114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b86e6622-02a6-43e0-9f3f-1399172f4e1d admin admin] The token cache is being invalidate because project c2bdfb3a940b4fa3b58a781d27c51563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:49.654508 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b86e6622-02a6-43e0-9f3f-1399172f4e1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:49.655151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b86e6622-02a6-43e0-9f3f-1399172f4e1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:49.656698 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1283/2580] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:49 2022] DELETE /identity/v3/projects/c2bdfb3a940b4fa3b58a781d27c51563 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:49.661269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad50c96-c29d-4fd1-8bdc-30fe7231cbb2 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:49.661554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad50c96-c29d-4fd1-8bdc-30fe7231cbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:49.661829 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad50c96-c29d-4fd1-8bdc-30fe7231cbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:49.710698 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8ad50c96-c29d-4fd1-8bdc-30fe7231cbb2 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:49.711883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8ad50c96-c29d-4fd1-8bdc-30fe7231cbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:49.718830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ae021b-0690-4312-8e0f-4df1c355f294 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:49.751105 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:49.773758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1284/2581] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:49.774944 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd66076-9819-48f5-b52c-d090500c1a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:49.776417 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd66076-9819-48f5-b52c-d090500c1a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:49.776694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd66076-9819-48f5-b52c-d090500c1a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:49.776993 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd66076-9819-48f5-b52c-d090500c1a7b admin admin] PATH_INFO: `/v3/users/9ab2b71222cb4a85958ee2d87fde2236` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:49.778189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd66076-9819-48f5-b52c-d090500c1a7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab2b71222cb4a85958ee2d87fde2236)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:49.780157 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd66076-9819-48f5-b52c-d090500c1a7b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:49.781892 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba75d2f-1985-4d82-ac8c-36466e7b8c65 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:49.782178 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba75d2f-1985-4d82-ac8c-36466e7b8c65 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:49.782461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba75d2f-1985-4d82-ac8c-36466e7b8c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:49.799815 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-cba75d2f-1985-4d82-ac8c-36466e7b8c65 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:49.801047 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cba75d2f-1985-4d82-ac8c-36466e7b8c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:49.809495 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1285/2582] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:49.813865 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba75d2f-1985-4d82-ac8c-36466e7b8c65 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:49.825047 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba75d2f-1985-4d82-ac8c-36466e7b8c65 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:49.826980 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ebd66076-9819-48f5-b52c-d090500c1a7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab2b71222cb4a85958ee2d87fde2236'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:49.827594 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ebd66076-9819-48f5-b52c-d090500c1a7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab2b71222cb4a85958ee2d87fde2236'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:49.829830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f65d1-4e9b-41f1-b452-3161ddb6d647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:49.830490 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f65d1-4e9b-41f1-b452-3161ddb6d647 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:49.830771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f65d1-4e9b-41f1-b452-3161ddb6d647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:49.831142 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25f65d1-4e9b-41f1-b452-3161ddb6d647 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:49.831805 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f65d1-4e9b-41f1-b452-3161ddb6d647 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:21:49.832116 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f65d1-4e9b-41f1-b452-3161ddb6d647 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:49.833753 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25f65d1-4e9b-41f1-b452-3161ddb6d647 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:49.845104 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ebd66076-9819-48f5-b52c-d090500c1a7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab2b71222cb4a85958ee2d87fde2236'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:49.848830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1286/2583] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:21:49.853003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f65d1-4e9b-41f1-b452-3161ddb6d647 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:49.853864 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ebd66076-9819-48f5-b52c-d090500c1a7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab2b71222cb4a85958ee2d87fde2236'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:49.854813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b25f65d1-4e9b-41f1-b452-3161ddb6d647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:49.864356 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-ebd66076-9819-48f5-b52c-d090500c1a7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab2b71222cb4a85958ee2d87fde2236'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:49.868817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25f65d1-4e9b-41f1-b452-3161ddb6d647 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:49.871185 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1298/2584] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:49 2022] DELETE /identity/v3/users/9ab2b71222cb4a85958ee2d87fde2236 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:21:49.874554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac6df9-6884-49e7-9103-158affee7be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:49.875108 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac6df9-6884-49e7-9103-158affee7be8 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:49.875391 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac6df9-6884-49e7-9103-158affee7be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:49.875660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac6df9-6884-49e7-9103-158affee7be8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:49.876355 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac6df9-6884-49e7-9103-158affee7be8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:49.878303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac6df9-6884-49e7-9103-158affee7be8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:49.910615 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1287/2585] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:49 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:21:49.914500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd66076-9819-48f5-b52c-d090500c1a7b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:49.987285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd66076-9819-48f5-b52c-d090500c1a7b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:50.058617 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6d1b8-5dcf-4c7f-9d83-4363a79ed854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:50.059584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6d1b8-5dcf-4c7f-9d83-4363a79ed854 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.060002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6d1b8-5dcf-4c7f-9d83-4363a79ed854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.060364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6d1b8-5dcf-4c7f-9d83-4363a79ed854 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.061591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6d1b8-5dcf-4c7f-9d83-4363a79ed854 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:50.063853 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d6d1b8-5dcf-4c7f-9d83-4363a79ed854 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:50.073230 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:50.086978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1299/2586] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:49 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:21:50.091321 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac6df9-6884-49e7-9103-158affee7be8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:50.103050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac6df9-6884-49e7-9103-158affee7be8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:50.108759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05285a4a-5554-456a-a037-5a3e8c724ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:50.109531 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05285a4a-5554-456a-a037-5a3e8c724ebe admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.109880 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05285a4a-5554-456a-a037-5a3e8c724ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.110526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05285a4a-5554-456a-a037-5a3e8c724ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.111015 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05285a4a-5554-456a-a037-5a3e8c724ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:50.112667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05285a4a-5554-456a-a037-5a3e8c724ebe admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:50.123174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1288/2587] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:50.127602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6d1b8-5dcf-4c7f-9d83-4363a79ed854 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:50.138306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6d1b8-5dcf-4c7f-9d83-4363a79ed854 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:50.143367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc46c2c-f8e3-43a1-bb24-05fe39686ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:50.144163 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc46c2c-f8e3-43a1-bb24-05fe39686ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.144579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc46c2c-f8e3-43a1-bb24-05fe39686ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.144852 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc46c2c-f8e3-43a1-bb24-05fe39686ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.145618 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc46c2c-f8e3-43a1-bb24-05fe39686ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:50.147193 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc46c2c-f8e3-43a1-bb24-05fe39686ffb admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:50.157781 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1300/2588] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:50.161843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05285a4a-5554-456a-a037-5a3e8c724ebe admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:50.173426 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05285a4a-5554-456a-a037-5a3e8c724ebe admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:50.178328 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5879c5c-aebc-4bd0-a286-9bfbe308c146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:50.179218 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5879c5c-aebc-4bd0-a286-9bfbe308c146 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.179499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5879c5c-aebc-4bd0-a286-9bfbe308c146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.179759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5879c5c-aebc-4bd0-a286-9bfbe308c146 admin admin] PATH_INFO: `/v3/projects/b1aabe811e4541d48455870976ba6ff2/users/a7a5fa8dec4f4968b28f3057cd13e580/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.194201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc46c2c-f8e3-43a1-bb24-05fe39686ffb admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:50.195833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5879c5c-aebc-4bd0-a286-9bfbe308c146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1aabe811e4541d48455870976ba6ff2, user_id=a7a5fa8dec4f4968b28f3057cd13e580, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:50.196519 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3fc46c2c-f8e3-43a1-bb24-05fe39686ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:50.198928 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5879c5c-aebc-4bd0-a286-9bfbe308c146 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:50.210024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc46c2c-f8e3-43a1-bb24-05fe39686ffb admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:50.215139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb508ce5-3064-447e-a0c3-47941987d4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:50.216131 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb508ce5-3064-447e-a0c3-47941987d4fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.216573 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb508ce5-3064-447e-a0c3-47941987d4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.216845 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb508ce5-3064-447e-a0c3-47941987d4fc admin admin] PATH_INFO: `/v3/projects/eaaf1448908e48fdacbd1b284571466b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.218189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb508ce5-3064-447e-a0c3-47941987d4fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaaf1448908e48fdacbd1b284571466b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:50.220328 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb508ce5-3064-447e-a0c3-47941987d4fc admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:50.222083 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1289/2589] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:50 2022] PUT /identity/v3/projects/b1aabe811e4541d48455870976ba6ff2/users/a7a5fa8dec4f4968b28f3057cd13e580/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:50.233360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8c815-d22b-4c09-9175-834a5aea4bf4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.233651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8c815-d22b-4c09-9175-834a5aea4bf4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.233928 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8c815-d22b-4c09-9175-834a5aea4bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.263282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-fee8c815-d22b-4c09-9175-834a5aea4bf4 None None] MFA Rules not processed for user `a7a5fa8dec4f4968b28f3057cd13e580`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:50.264574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fee8c815-d22b-4c09-9175-834a5aea4bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:50.274272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:50.277001 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb508ce5-3064-447e-a0c3-47941987d4fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaaf1448908e48fdacbd1b284571466b', 'request_id': 'req-bb508ce5-3064-447e-a0c3-47941987d4fc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:50.284061 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb508ce5-3064-447e-a0c3-47941987d4fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaaf1448908e48fdacbd1b284571466b', 'request_id': 'req-bb508ce5-3064-447e-a0c3-47941987d4fc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:50.289849 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb508ce5-3064-447e-a0c3-47941987d4fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaaf1448908e48fdacbd1b284571466b', 'request_id': 'req-bb508ce5-3064-447e-a0c3-47941987d4fc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:50.302320 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb508ce5-3064-447e-a0c3-47941987d4fc admin admin] The token cache is being invalidate because project eaaf1448908e48fdacbd1b284571466b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:50.302603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb508ce5-3064-447e-a0c3-47941987d4fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:50.303235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-bb508ce5-3064-447e-a0c3-47941987d4fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:50.304812 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1301/2590] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:50 2022] DELETE /identity/v3/projects/eaaf1448908e48fdacbd1b284571466b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:21:50.309951 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d7692-aeb0-4567-9d96-abd9cf2055af None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.310249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d7692-aeb0-4567-9d96-abd9cf2055af None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.310512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662d7692-aeb0-4567-9d96-abd9cf2055af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.328618 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-662d7692-aeb0-4567-9d96-abd9cf2055af None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:50.329847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-662d7692-aeb0-4567-9d96-abd9cf2055af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:50.337539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1290/2591] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:50.348690 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8c815-d22b-4c09-9175-834a5aea4bf4 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:50.350056 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fee8c815-d22b-4c09-9175-834a5aea4bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:50.391445 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1302/2592] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:50.399403 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d025fee-66b4-4038-ba10-af1fe3a5bb4a None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.399984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d025fee-66b4-4038-ba10-af1fe3a5bb4a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.400266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d025fee-66b4-4038-ba10-af1fe3a5bb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.417001 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8c815-d22b-4c09-9175-834a5aea4bf4 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:50.419747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5d025fee-66b4-4038-ba10-af1fe3a5bb4a None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:50.421000 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5d025fee-66b4-4038-ba10-af1fe3a5bb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:50.421374 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0c9c5-110a-4086-86f8-fccf4169fc48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:50.422090 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0c9c5-110a-4086-86f8-fccf4169fc48 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.422370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0c9c5-110a-4086-86f8-fccf4169fc48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.422641 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd0c9c5-110a-4086-86f8-fccf4169fc48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.429292 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1303/2593] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:50.431568 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0c9c5-110a-4086-86f8-fccf4169fc48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:50.433637 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd0c9c5-110a-4086-86f8-fccf4169fc48 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:50.434050 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d025fee-66b4-4038-ba10-af1fe3a5bb4a None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:50.443398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d025fee-66b4-4038-ba10-af1fe3a5bb4a None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:50.445561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1291/2594] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:21:50.448976 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467afff6-54a8-4e7f-96e8-4e42f80ed049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:50.449723 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467afff6-54a8-4e7f-96e8-4e42f80ed049 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.450070 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467afff6-54a8-4e7f-96e8-4e42f80ed049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.450438 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467afff6-54a8-4e7f-96e8-4e42f80ed049 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.451243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467afff6-54a8-4e7f-96e8-4e42f80ed049 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:21:50.451625 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467afff6-54a8-4e7f-96e8-4e42f80ed049 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:50.453532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467afff6-54a8-4e7f-96e8-4e42f80ed049 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:50.467018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1304/2595] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:21:50.471427 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0c9c5-110a-4086-86f8-fccf4169fc48 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:50.474830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467afff6-54a8-4e7f-96e8-4e42f80ed049 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:50.476484 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-467afff6-54a8-4e7f-96e8-4e42f80ed049 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:50.484809 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd0c9c5-110a-4086-86f8-fccf4169fc48 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:50.490002 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc850c22-6804-45dd-9f4b-8b122ef3e745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:50.490389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467afff6-54a8-4e7f-96e8-4e42f80ed049 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:50.490654 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc850c22-6804-45dd-9f4b-8b122ef3e745 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.490931 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc850c22-6804-45dd-9f4b-8b122ef3e745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.491203 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc850c22-6804-45dd-9f4b-8b122ef3e745 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.491900 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc850c22-6804-45dd-9f4b-8b122ef3e745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:50.493825 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc850c22-6804-45dd-9f4b-8b122ef3e745 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:50.495799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0abd0-14a5-436a-87e7-9640bff312f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:50.496670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0abd0-14a5-436a-87e7-9640bff312f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.496978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0abd0-14a5-436a-87e7-9640bff312f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.497259 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0abd0-14a5-436a-87e7-9640bff312f7 admin admin] PATH_INFO: `/v3/users/a7a5fa8dec4f4968b28f3057cd13e580` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.498493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0abd0-14a5-436a-87e7-9640bff312f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a5fa8dec4f4968b28f3057cd13e580)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:50.500406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0abd0-14a5-436a-87e7-9640bff312f7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:50.526002 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1292/2596] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:50 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:21:50.527259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc850c22-6804-45dd-9f4b-8b122ef3e745 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:50.552087 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:INSERT=2,keystone:SELECT=85,keystone:DELETE=13 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:50.560685 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-79f0abd0-14a5-436a-87e7-9640bff312f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a5fa8dec4f4968b28f3057cd13e580'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:50.562163 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-79f0abd0-14a5-436a-87e7-9640bff312f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a5fa8dec4f4968b28f3057cd13e580'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:50.577273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-79f0abd0-14a5-436a-87e7-9640bff312f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a5fa8dec4f4968b28f3057cd13e580'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:50.585327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-79f0abd0-14a5-436a-87e7-9640bff312f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a5fa8dec4f4968b28f3057cd13e580'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:50.594822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-79f0abd0-14a5-436a-87e7-9640bff312f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a5fa8dec4f4968b28f3057cd13e580'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:50.598831 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc850c22-6804-45dd-9f4b-8b122ef3e745 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:50.602380 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1305/2597] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:50 2022] DELETE /identity/v3/users/a7a5fa8dec4f4968b28f3057cd13e580 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:50.606602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e13406-0711-47b7-810c-3d0d282a0b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:50.607281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e13406-0711-47b7-810c-3d0d282a0b6f admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.607500 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28034dd-295c-4a74-93d4-985841944870 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.607614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e13406-0711-47b7-810c-3d0d282a0b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.607800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28034dd-295c-4a74-93d4-985841944870 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.607903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e13406-0711-47b7-810c-3d0d282a0b6f admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.608092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28034dd-295c-4a74-93d4-985841944870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.608644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e13406-0711-47b7-810c-3d0d282a0b6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:50.610614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e13406-0711-47b7-810c-3d0d282a0b6f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:50.627913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-e28034dd-295c-4a74-93d4-985841944870 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:50.629138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e28034dd-295c-4a74-93d4-985841944870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:50.635065 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1293/2598] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:50 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:21:50.636649 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e13406-0711-47b7-810c-3d0d282a0b6f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:50.638520 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-12e13406-0711-47b7-810c-3d0d282a0b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:50.652116 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e13406-0711-47b7-810c-3d0d282a0b6f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:50.672375 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:50.703958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1306/2599] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:50.714218 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28034dd-295c-4a74-93d4-985841944870 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:50.715635 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e28034dd-295c-4a74-93d4-985841944870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:50.721576 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c638197-8a6c-4b5a-a01d-77d7a483adf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:50.722135 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c638197-8a6c-4b5a-a01d-77d7a483adf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.722411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c638197-8a6c-4b5a-a01d-77d7a483adf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.722688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c638197-8a6c-4b5a-a01d-77d7a483adf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.723458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c638197-8a6c-4b5a-a01d-77d7a483adf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:50.725142 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c638197-8a6c-4b5a-a01d-77d7a483adf8 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:50.739990 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1294/2600] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:50.741195 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c638197-8a6c-4b5a-a01d-77d7a483adf8 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:50.752810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c638197-8a6c-4b5a-a01d-77d7a483adf8 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:50.758400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d2843-a7e6-48d5-b50e-0e0ecc563406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:50.758716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d2843-a7e6-48d5-b50e-0e0ecc563406 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.759110 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d2843-a7e6-48d5-b50e-0e0ecc563406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.759411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305d2843-a7e6-48d5-b50e-0e0ecc563406 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.760182 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d2843-a7e6-48d5-b50e-0e0ecc563406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:50.761844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305d2843-a7e6-48d5-b50e-0e0ecc563406 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:50.772494 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1295/2601] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:50.776567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d2843-a7e6-48d5-b50e-0e0ecc563406 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:50.786581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305d2843-a7e6-48d5-b50e-0e0ecc563406 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:50.787917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28034dd-295c-4a74-93d4-985841944870 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:50.792138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b1775-5c23-4917-90f7-45d0f510ca80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:50.792387 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126511c-d265-43d0-b2dc-18121e371313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:50.792778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b1775-5c23-4917-90f7-45d0f510ca80 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.793079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b1775-5c23-4917-90f7-45d0f510ca80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.793346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b1775-5c23-4917-90f7-45d0f510ca80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.793655 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126511c-d265-43d0-b2dc-18121e371313 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.794044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126511c-d265-43d0-b2dc-18121e371313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.794386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126511c-d265-43d0-b2dc-18121e371313 admin admin] PATH_INFO: `/v3/projects/7412eadd9f0b4288b94e59e7b34717cc/users/deb748d89f2b44b8b0fbd8e404c7a0f6/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.804572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b1775-5c23-4917-90f7-45d0f510ca80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:50.805970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527b1775-5c23-4917-90f7-45d0f510ca80 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:50.811566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126511c-d265-43d0-b2dc-18121e371313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7412eadd9f0b4288b94e59e7b34717cc, user_id=deb748d89f2b44b8b0fbd8e404c7a0f6, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:50.814458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126511c-d265-43d0-b2dc-18121e371313 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:50.817892 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1307/2602] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:21:50.833813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1296/2603] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:50 2022] PUT /identity/v3/projects/7412eadd9f0b4288b94e59e7b34717cc/users/deb748d89f2b44b8b0fbd8e404c7a0f6/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:21:50.844583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22e3d5-062a-4f88-b438-80dc2316836c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:50.844796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22e3d5-062a-4f88-b438-80dc2316836c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:50.845115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d22e3d5-062a-4f88-b438-80dc2316836c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:50.882615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-9d22e3d5-062a-4f88-b438-80dc2316836c None None] MFA Rules not processed for user `deb748d89f2b44b8b0fbd8e404c7a0f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:50.883961 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9d22e3d5-062a-4f88-b438-80dc2316836c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:50.940904 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:50.966783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1308/2604] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:50.971431 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126511c-d265-43d0-b2dc-18121e371313 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:51.070609 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126511c-d265-43d0-b2dc-18121e371313 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:51.076017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c45414-5486-4d99-bad0-45d48c734c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:51.076788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c45414-5486-4d99-bad0-45d48c734c9a admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.077106 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c45414-5486-4d99-bad0-45d48c734c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.077381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c45414-5486-4d99-bad0-45d48c734c9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.078076 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c45414-5486-4d99-bad0-45d48c734c9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:51.079980 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c45414-5486-4d99-bad0-45d48c734c9a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:51.113364 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1297/2605] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:50 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:21:51.115182 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22e3d5-062a-4f88-b438-80dc2316836c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:51.127412 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d22e3d5-062a-4f88-b438-80dc2316836c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:51.190367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c45414-5486-4d99-bad0-45d48c734c9a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:51.192739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-68c45414-5486-4d99-bad0-45d48c734c9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:51.204570 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bbb5c-c2df-4376-ae22-7483796ac42d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:51.206016 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bbb5c-c2df-4376-ae22-7483796ac42d admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.206478 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bbb5c-c2df-4376-ae22-7483796ac42d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.206916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bbb5c-c2df-4376-ae22-7483796ac42d admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.207936 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bbb5c-c2df-4376-ae22-7483796ac42d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:51.211418 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2bbb5c-c2df-4376-ae22-7483796ac42d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:51.259245 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1309/2606] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:51 2022] POST /identity/v3/users => generated 299 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:51.262424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bbb5c-c2df-4376-ae22-7483796ac42d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:51.274573 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bbb5c-c2df-4376-ae22-7483796ac42d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:51.280742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5e0c2-aaa2-422d-878f-6fa12aa43fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:51.281509 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5e0c2-aaa2-422d-878f-6fa12aa43fae admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.281788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5e0c2-aaa2-422d-878f-6fa12aa43fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.282058 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5e0c2-aaa2-422d-878f-6fa12aa43fae admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.282791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5e0c2-aaa2-422d-878f-6fa12aa43fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:51.284481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5e0c2-aaa2-422d-878f-6fa12aa43fae admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:51.286210 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c45414-5486-4d99-bad0-45d48c734c9a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:51.290885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f3dd86-138f-4725-9755-4811a162e63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:51.291869 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3dd86-138f-4725-9755-4811a162e63e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.292130 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3dd86-138f-4725-9755-4811a162e63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.292428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f3dd86-138f-4725-9755-4811a162e63e admin admin] PATH_INFO: `/v3/projects/b1aabe811e4541d48455870976ba6ff2` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.293674 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3dd86-138f-4725-9755-4811a162e63e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1aabe811e4541d48455870976ba6ff2)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:51.295577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f3dd86-138f-4725-9755-4811a162e63e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:51.297527 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1310/2607] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:51.301589 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5e0c2-aaa2-422d-878f-6fa12aa43fae admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:51.307809 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:51.313585 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5e0c2-aaa2-422d-878f-6fa12aa43fae admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:51.319440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aabaf-86bd-41b4-823d-5b64a5cc8bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:51.320649 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aabaf-86bd-41b4-823d-5b64a5cc8bfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.320913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aabaf-86bd-41b4-823d-5b64a5cc8bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.321241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435aabaf-86bd-41b4-823d-5b64a5cc8bfa admin admin] PATH_INFO: `/v3/projects/9447546da8e242b39618b961f5d72f86/users/bedf2772e74a402d896f0d0cfa5e4def/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.338153 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aabaf-86bd-41b4-823d-5b64a5cc8bfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=9447546da8e242b39618b961f5d72f86, user_id=bedf2772e74a402d896f0d0cfa5e4def, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:51.341368 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435aabaf-86bd-41b4-823d-5b64a5cc8bfa admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:51.355966 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1311/2608] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:51 2022] PUT /identity/v3/projects/9447546da8e242b39618b961f5d72f86/users/bedf2772e74a402d896f0d0cfa5e4def/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:21:51.357405 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-08f3dd86-138f-4725-9755-4811a162e63e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1aabe811e4541d48455870976ba6ff2', 'request_id': 'req-08f3dd86-138f-4725-9755-4811a162e63e'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:51.359826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aabaf-86bd-41b4-823d-5b64a5cc8bfa admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:51.365789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-08f3dd86-138f-4725-9755-4811a162e63e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1aabe811e4541d48455870976ba6ff2', 'request_id': 'req-08f3dd86-138f-4725-9755-4811a162e63e'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:51.374830 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-08f3dd86-138f-4725-9755-4811a162e63e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1aabe811e4541d48455870976ba6ff2', 'request_id': 'req-08f3dd86-138f-4725-9755-4811a162e63e'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:51.387011 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-08f3dd86-138f-4725-9755-4811a162e63e admin admin] The token cache is being invalidate because project b1aabe811e4541d48455870976ba6ff2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:51.387296 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-08f3dd86-138f-4725-9755-4811a162e63e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:51.387960 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-08f3dd86-138f-4725-9755-4811a162e63e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:51.389736 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1298/2609] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:51 2022] DELETE /identity/v3/projects/b1aabe811e4541d48455870976ba6ff2 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:21:51.395111 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0119f97-c53b-443f-acd4-67cc1ff67cb0 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.395717 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0119f97-c53b-443f-acd4-67cc1ff67cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.396071 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0119f97-c53b-443f-acd4-67cc1ff67cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.419305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e0119f97-c53b-443f-acd4-67cc1ff67cb0 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:51.421020 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e0119f97-c53b-443f-acd4-67cc1ff67cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:51.430004 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1299/2610] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:51.430406 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435aabaf-86bd-41b4-823d-5b64a5cc8bfa admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:51.435100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe49b7-6ad8-4127-bd32-76091ee253b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:51.435770 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe49b7-6ad8-4127-bd32-76091ee253b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.436052 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe49b7-6ad8-4127-bd32-76091ee253b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.436328 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fe49b7-6ad8-4127-bd32-76091ee253b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.437530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe49b7-6ad8-4127-bd32-76091ee253b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:51.438528 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b477d-bcdb-4a5f-a47e-cd0f22642cee None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.438815 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b477d-bcdb-4a5f-a47e-cd0f22642cee None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.439092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b477d-bcdb-4a5f-a47e-cd0f22642cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.439211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fe49b7-6ad8-4127-bd32-76091ee253b7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:51.453216 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1312/2611] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:51.454449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe49b7-6ad8-4127-bd32-76091ee253b7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:51.456506 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-09fe49b7-6ad8-4127-bd32-76091ee253b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:51.458950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-409b477d-bcdb-4a5f-a47e-cd0f22642cee None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:51.460132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-409b477d-bcdb-4a5f-a47e-cd0f22642cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:51.469825 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1300/2612] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:51.471756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fe49b7-6ad8-4127-bd32-76091ee253b7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:51.473744 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b477d-bcdb-4a5f-a47e-cd0f22642cee None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:51.477041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a77d48-932e-477c-ba39-8fc0a75195dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:51.478009 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a77d48-932e-477c-ba39-8fc0a75195dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.478298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a77d48-932e-477c-ba39-8fc0a75195dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.478603 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a77d48-932e-477c-ba39-8fc0a75195dd admin admin] PATH_INFO: `/v3/projects/9447546da8e242b39618b961f5d72f86/users/bedf2772e74a402d896f0d0cfa5e4def/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.480730 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a77d48-932e-477c-ba39-8fc0a75195dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9447546da8e242b39618b961f5d72f86, user_id=bedf2772e74a402d896f0d0cfa5e4def, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:51.483499 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a77d48-932e-477c-ba39-8fc0a75195dd admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:51.483962 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b477d-bcdb-4a5f-a47e-cd0f22642cee None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:51.489893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ee5ac-63a6-41d6-b4b8-b1ccc58e5b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:51.489893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ee5ac-63a6-41d6-b4b8-b1ccc58e5b5f admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.489893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ee5ac-63a6-41d6-b4b8-b1ccc58e5b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.490215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ee5ac-63a6-41d6-b4b8-b1ccc58e5b5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.490739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ee5ac-63a6-41d6-b4b8-b1ccc58e5b5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:21:51.491050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ee5ac-63a6-41d6-b4b8-b1ccc58e5b5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:51.492634 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ee5ac-63a6-41d6-b4b8-b1ccc58e5b5f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:51.495609 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1313/2613] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:51 2022] PUT /identity/v3/projects/9447546da8e242b39618b961f5d72f86/users/bedf2772e74a402d896f0d0cfa5e4def/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:51.505948 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1301/2614] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:21:51.507240 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587a940-eb5b-4e32-904b-6cc2982fd3e2 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.507872 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587a940-eb5b-4e32-904b-6cc2982fd3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.508435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587a940-eb5b-4e32-904b-6cc2982fd3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.511303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ee5ac-63a6-41d6-b4b8-b1ccc58e5b5f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:51.522359 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ee5ac-63a6-41d6-b4b8-b1ccc58e5b5f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:51.540868 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-f587a940-eb5b-4e32-904b-6cc2982fd3e2 None None] MFA Rules not processed for user `bedf2772e74a402d896f0d0cfa5e4def`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:51.542157 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f587a940-eb5b-4e32-904b-6cc2982fd3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:51.560203 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:51.586750 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead48c8-ad4a-4281-8407-ef7d9f6243a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:51.587411 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead48c8-ad4a-4281-8407-ef7d9f6243a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.587687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead48c8-ad4a-4281-8407-ef7d9f6243a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.587957 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ead48c8-ad4a-4281-8407-ef7d9f6243a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.589217 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead48c8-ad4a-4281-8407-ef7d9f6243a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:51.590619 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ead48c8-ad4a-4281-8407-ef7d9f6243a3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:51.617627 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1302/2615] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:51 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:21:51.622372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead48c8-ad4a-4281-8407-ef7d9f6243a3 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:51.632434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1314/2616] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:51.640698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587a940-eb5b-4e32-904b-6cc2982fd3e2 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:51.642465 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f587a940-eb5b-4e32-904b-6cc2982fd3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:51.690280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:51.694960 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ead48c8-ad4a-4281-8407-ef7d9f6243a3 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:51.706855 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c705e30-98c8-4c58-b059-349919d77096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:51.707528 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c705e30-98c8-4c58-b059-349919d77096 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.707808 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c705e30-98c8-4c58-b059-349919d77096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.708622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c705e30-98c8-4c58-b059-349919d77096 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.709540 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c705e30-98c8-4c58-b059-349919d77096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:51.711463 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c705e30-98c8-4c58-b059-349919d77096 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:51.716499 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f587a940-eb5b-4e32-904b-6cc2982fd3e2 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:51.737268 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1303/2617] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:51 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:21:51.742049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c705e30-98c8-4c58-b059-349919d77096 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:51.754428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c705e30-98c8-4c58-b059-349919d77096 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:51.759847 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ce8d2-a6ad-4138-8998-bdd9d7009e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:51.761134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ce8d2-a6ad-4138-8998-bdd9d7009e0f admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.762894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ce8d2-a6ad-4138-8998-bdd9d7009e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.762894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ce8d2-a6ad-4138-8998-bdd9d7009e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.762894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ce8d2-a6ad-4138-8998-bdd9d7009e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:51.764123 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ce8d2-a6ad-4138-8998-bdd9d7009e0f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:51.775751 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1304/2618] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:21:51.777156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a1ae6-35d0-49cd-9bb6-0e15cb68cc4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:51.777838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a1ae6-35d0-49cd-9bb6-0e15cb68cc4c service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.778122 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a1ae6-35d0-49cd-9bb6-0e15cb68cc4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.778412 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a1ae6-35d0-49cd-9bb6-0e15cb68cc4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.779590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ce8d2-a6ad-4138-8998-bdd9d7009e0f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:51.789936 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ce8d2-a6ad-4138-8998-bdd9d7009e0f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:51.794834 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22091d8b-91f4-456f-8372-1b8e736e3576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:51.795500 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22091d8b-91f4-456f-8372-1b8e736e3576 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.795783 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22091d8b-91f4-456f-8372-1b8e736e3576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.796062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22091d8b-91f4-456f-8372-1b8e736e3576 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.796815 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22091d8b-91f4-456f-8372-1b8e736e3576 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:51.798438 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22091d8b-91f4-456f-8372-1b8e736e3576 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:51.851347 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a1ae6-35d0-49cd-9bb6-0e15cb68cc4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:51.852810 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a1ae6-35d0-49cd-9bb6-0e15cb68cc4c service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:51.864698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1315/2619] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:51 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:21:51.935468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a1ae6-35d0-49cd-9bb6-0e15cb68cc4c service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:51.939668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1305/2620] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:51.941895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22091d8b-91f4-456f-8372-1b8e736e3576 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:51.946398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a1ae6-35d0-49cd-9bb6-0e15cb68cc4c service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:51.951553 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cce7bd-824b-47b3-b50c-85c7c133c838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:51.952420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cce7bd-824b-47b3-b50c-85c7c133c838 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.952697 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cce7bd-824b-47b3-b50c-85c7c133c838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.952832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22091d8b-91f4-456f-8372-1b8e736e3576 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:51.953005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cce7bd-824b-47b3-b50c-85c7c133c838 admin admin] PATH_INFO: `/v3/users/deb748d89f2b44b8b0fbd8e404c7a0f6` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.954267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cce7bd-824b-47b3-b50c-85c7c133c838 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deb748d89f2b44b8b0fbd8e404c7a0f6)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:51.956098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cce7bd-824b-47b3-b50c-85c7c133c838 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:51.957440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b282c8ec-6600-4399-ab7d-08045574da4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:51.958297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282c8ec-6600-4399-ab7d-08045574da4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:51.958571 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282c8ec-6600-4399-ab7d-08045574da4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:51.958838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b282c8ec-6600-4399-ab7d-08045574da4e admin admin] PATH_INFO: `/v3/projects/bf1ba07f5cc949d7b269cf95f48a17f6/users/eb3798d83a6644daa6f42f8456466cea/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:51.970595 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:51.973938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282c8ec-6600-4399-ab7d-08045574da4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf1ba07f5cc949d7b269cf95f48a17f6, user_id=eb3798d83a6644daa6f42f8456466cea, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:51.976769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b282c8ec-6600-4399-ab7d-08045574da4e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:51.990951 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1306/2621] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:51 2022] PUT /identity/v3/projects/bf1ba07f5cc949d7b269cf95f48a17f6/users/eb3798d83a6644daa6f42f8456466cea/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:21:52.002783 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860909e-d003-47f6-be02-3e880f733348 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:52.003075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860909e-d003-47f6-be02-3e880f733348 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:52.003345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860909e-d003-47f6-be02-3e880f733348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:52.007796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-96cce7bd-824b-47b3-b50c-85c7c133c838 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deb748d89f2b44b8b0fbd8e404c7a0f6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:52.008463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-96cce7bd-824b-47b3-b50c-85c7c133c838 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deb748d89f2b44b8b0fbd8e404c7a0f6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:52.021767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-96cce7bd-824b-47b3-b50c-85c7c133c838 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deb748d89f2b44b8b0fbd8e404c7a0f6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:52.028615 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-96cce7bd-824b-47b3-b50c-85c7c133c838 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deb748d89f2b44b8b0fbd8e404c7a0f6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:52.030289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-1860909e-d003-47f6-be02-3e880f733348 None None] MFA Rules not processed for user `eb3798d83a6644daa6f42f8456466cea`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:52.031530 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1860909e-d003-47f6-be02-3e880f733348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:52.036462 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-96cce7bd-824b-47b3-b50c-85c7c133c838 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deb748d89f2b44b8b0fbd8e404c7a0f6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:52.043087 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1316/2622] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:51 2022] DELETE /identity/v3/users/deb748d89f2b44b8b0fbd8e404c7a0f6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:21:52.047394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbd23c-c7bc-41ae-b9b6-4c3054e32bbf None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:52.047677 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbd23c-c7bc-41ae-b9b6-4c3054e32bbf None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:52.047965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedbd23c-c7bc-41ae-b9b6-4c3054e32bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:52.065305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-eedbd23c-c7bc-41ae-b9b6-4c3054e32bbf None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:52.066503 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-eedbd23c-c7bc-41ae-b9b6-4c3054e32bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:52.098828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1307/2623] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:52.110090 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860909e-d003-47f6-be02-3e880f733348 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:52.111464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1860909e-d003-47f6-be02-3e880f733348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:52.128392 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1317/2624] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:52.137874 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbd23c-c7bc-41ae-b9b6-4c3054e32bbf None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:52.139315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-eedbd23c-c7bc-41ae-b9b6-4c3054e32bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:52.171043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860909e-d003-47f6-be02-3e880f733348 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:52.195951 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedbd23c-c7bc-41ae-b9b6-4c3054e32bbf None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:52.199624 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805f510-bb0e-4329-971e-faa2e7019330 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:52.200306 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805f510-bb0e-4329-971e-faa2e7019330 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:52.200593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805f510-bb0e-4329-971e-faa2e7019330 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:52.200872 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805f510-bb0e-4329-971e-faa2e7019330 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:52.210606 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805f510-bb0e-4329-971e-faa2e7019330 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:52.212137 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805f510-bb0e-4329-971e-faa2e7019330 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:52.213267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:52.219622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e683866-4bc3-4af2-b692-155ac67b97e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:52.220467 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e683866-4bc3-4af2-b692-155ac67b97e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:52.220958 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e683866-4bc3-4af2-b692-155ac67b97e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:52.221235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e683866-4bc3-4af2-b692-155ac67b97e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:52.222665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1318/2625] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:21:52.284594 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e683866-4bc3-4af2-b692-155ac67b97e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:52.286565 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e683866-4bc3-4af2-b692-155ac67b97e4 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:52.301392 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1308/2626] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:21:52.569557 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805f510-bb0e-4329-971e-faa2e7019330 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:52.571640 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d805f510-bb0e-4329-971e-faa2e7019330 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:52.603160 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805f510-bb0e-4329-971e-faa2e7019330 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:52.610158 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfaa7a-ba41-4a09-aa20-a343db999045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:52.611273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfaa7a-ba41-4a09-aa20-a343db999045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:52.611658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfaa7a-ba41-4a09-aa20-a343db999045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:52.611989 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbfaa7a-ba41-4a09-aa20-a343db999045 admin admin] PATH_INFO: `/v3/projects/7412eadd9f0b4288b94e59e7b34717cc` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:52.613489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbfaa7a-ba41-4a09-aa20-a343db999045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7412eadd9f0b4288b94e59e7b34717cc)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:52.615565 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbfaa7a-ba41-4a09-aa20-a343db999045 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:52.656283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:52.673735 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9dbfaa7a-ba41-4a09-aa20-a343db999045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7412eadd9f0b4288b94e59e7b34717cc', 'request_id': 'req-9dbfaa7a-ba41-4a09-aa20-a343db999045'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:52.680345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9dbfaa7a-ba41-4a09-aa20-a343db999045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7412eadd9f0b4288b94e59e7b34717cc', 'request_id': 'req-9dbfaa7a-ba41-4a09-aa20-a343db999045'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:52.686967 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9dbfaa7a-ba41-4a09-aa20-a343db999045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7412eadd9f0b4288b94e59e7b34717cc', 'request_id': 'req-9dbfaa7a-ba41-4a09-aa20-a343db999045'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:52.708707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9dbfaa7a-ba41-4a09-aa20-a343db999045 admin admin] The token cache is being invalidate because project 7412eadd9f0b4288b94e59e7b34717cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:52.709033 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9dbfaa7a-ba41-4a09-aa20-a343db999045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:52.709673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9dbfaa7a-ba41-4a09-aa20-a343db999045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:52.711251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1319/2627] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:52 2022] DELETE /identity/v3/projects/7412eadd9f0b4288b94e59e7b34717cc => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:52.715605 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e683866-4bc3-4af2-b692-155ac67b97e4 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:52.726685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e683866-4bc3-4af2-b692-155ac67b97e4 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:52.786070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f331d10-b3bc-46c9-9489-02f47f22f9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:52.786942 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f331d10-b3bc-46c9-9489-02f47f22f9f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:52.787224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f331d10-b3bc-46c9-9489-02f47f22f9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:52.787499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f331d10-b3bc-46c9-9489-02f47f22f9f3 admin admin] PATH_INFO: `/v3/users/bedf2772e74a402d896f0d0cfa5e4def` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:52.788681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f331d10-b3bc-46c9-9489-02f47f22f9f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bedf2772e74a402d896f0d0cfa5e4def)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:52.790758 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f331d10-b3bc-46c9-9489-02f47f22f9f3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:52.835954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2f331d10-b3bc-46c9-9489-02f47f22f9f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bedf2772e74a402d896f0d0cfa5e4def'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:52.836599 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2f331d10-b3bc-46c9-9489-02f47f22f9f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bedf2772e74a402d896f0d0cfa5e4def'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:52.849079 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2f331d10-b3bc-46c9-9489-02f47f22f9f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bedf2772e74a402d896f0d0cfa5e4def'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:52.855977 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2f331d10-b3bc-46c9-9489-02f47f22f9f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bedf2772e74a402d896f0d0cfa5e4def'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:52.863369 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2f331d10-b3bc-46c9-9489-02f47f22f9f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bedf2772e74a402d896f0d0cfa5e4def'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:52.869475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1309/2628] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:52 2022] DELETE /identity/v3/users/bedf2772e74a402d896f0d0cfa5e4def => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:53.138581 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfaa7a-ba41-4a09-aa20-a343db999045 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:53.140044 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-9dbfaa7a-ba41-4a09-aa20-a343db999045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:53.153469 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbfaa7a-ba41-4a09-aa20-a343db999045 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:53.209083 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dd50db-d884-48d8-914c-b1fc56b1a491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:53.209942 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd50db-d884-48d8-914c-b1fc56b1a491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:53.210222 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd50db-d884-48d8-914c-b1fc56b1a491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:53.210493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dd50db-d884-48d8-914c-b1fc56b1a491 admin admin] PATH_INFO: `/v3/projects/9447546da8e242b39618b961f5d72f86` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:53.211655 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd50db-d884-48d8-914c-b1fc56b1a491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9447546da8e242b39618b961f5d72f86)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:53.213584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dd50db-d884-48d8-914c-b1fc56b1a491 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:53.259443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91dd50db-d884-48d8-914c-b1fc56b1a491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9447546da8e242b39618b961f5d72f86', 'request_id': 'req-91dd50db-d884-48d8-914c-b1fc56b1a491'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:53.266079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91dd50db-d884-48d8-914c-b1fc56b1a491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9447546da8e242b39618b961f5d72f86', 'request_id': 'req-91dd50db-d884-48d8-914c-b1fc56b1a491'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:53.271633 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91dd50db-d884-48d8-914c-b1fc56b1a491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9447546da8e242b39618b961f5d72f86', 'request_id': 'req-91dd50db-d884-48d8-914c-b1fc56b1a491'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:53.283880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91dd50db-d884-48d8-914c-b1fc56b1a491 admin admin] The token cache is being invalidate because project 9447546da8e242b39618b961f5d72f86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:53.284194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91dd50db-d884-48d8-914c-b1fc56b1a491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:53.284810 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-91dd50db-d884-48d8-914c-b1fc56b1a491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:53.286365 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1320/2629] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:53 2022] DELETE /identity/v3/projects/9447546da8e242b39618b961f5d72f86 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:53.291207 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dcb63-24f5-4105-a84b-0bd1022f2c8d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:53.291497 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dcb63-24f5-4105-a84b-0bd1022f2c8d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:53.291776 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dcb63-24f5-4105-a84b-0bd1022f2c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:53.309958 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-a90dcb63-24f5-4105-a84b-0bd1022f2c8d None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:53.311152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a90dcb63-24f5-4105-a84b-0bd1022f2c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:53.324351 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:53.370497 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1310/2630] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:53.378013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37be534-278c-44e2-aa41-845ffc3f9d3a None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:53.378301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37be534-278c-44e2-aa41-845ffc3f9d3a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:53.378575 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37be534-278c-44e2-aa41-845ffc3f9d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:53.397458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-a37be534-278c-44e2-aa41-845ffc3f9d3a None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:53.398584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a37be534-278c-44e2-aa41-845ffc3f9d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:53.406346 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1321/2631] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:53.410264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90dcb63-24f5-4105-a84b-0bd1022f2c8d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:53.419735 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90dcb63-24f5-4105-a84b-0bd1022f2c8d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:53.424233 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58420c-ffa5-4180-9bae-00f34e6644c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:53.424876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58420c-ffa5-4180-9bae-00f34e6644c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:53.425192 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58420c-ffa5-4180-9bae-00f34e6644c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:53.425468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58420c-ffa5-4180-9bae-00f34e6644c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:53.426192 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58420c-ffa5-4180-9bae-00f34e6644c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:21:53.426499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58420c-ffa5-4180-9bae-00f34e6644c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:53.428093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58420c-ffa5-4180-9bae-00f34e6644c4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:53.440046 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1311/2632] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:21:53.444362 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37be534-278c-44e2-aa41-845ffc3f9d3a None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:53.454712 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37be534-278c-44e2-aa41-845ffc3f9d3a None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:53.459285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed248bd3-51fb-489f-a9d1-530b173216b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:53.459924 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed248bd3-51fb-489f-a9d1-530b173216b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:53.460221 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed248bd3-51fb-489f-a9d1-530b173216b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:53.460471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed248bd3-51fb-489f-a9d1-530b173216b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:53.461195 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed248bd3-51fb-489f-a9d1-530b173216b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:53.463064 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed248bd3-51fb-489f-a9d1-530b173216b8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:53.489156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1322/2633] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:53 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:53.493257 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58420c-ffa5-4180-9bae-00f34e6644c4 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:53.503554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58420c-ffa5-4180-9bae-00f34e6644c4 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:53.560537 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389540e-3827-4e32-b78d-a46f34af0207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:53.561335 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389540e-3827-4e32-b78d-a46f34af0207 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:53.561613 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389540e-3827-4e32-b78d-a46f34af0207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:53.561883 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389540e-3827-4e32-b78d-a46f34af0207 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:53.562621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389540e-3827-4e32-b78d-a46f34af0207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:53.564518 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389540e-3827-4e32-b78d-a46f34af0207 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:53.584678 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1312/2634] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:53 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:53.588885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed248bd3-51fb-489f-a9d1-530b173216b8 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:53.598969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed248bd3-51fb-489f-a9d1-530b173216b8 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:53.604529 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f38d-32b5-42bf-b85d-17bd0af3d41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:53.605507 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f38d-32b5-42bf-b85d-17bd0af3d41d admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:53.605793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f38d-32b5-42bf-b85d-17bd0af3d41d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:53.606092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab3f38d-32b5-42bf-b85d-17bd0af3d41d admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:53.606827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3f38d-32b5-42bf-b85d-17bd0af3d41d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:53.608425 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab3f38d-32b5-42bf-b85d-17bd0af3d41d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:53.619452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1323/2635] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:53.623937 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389540e-3827-4e32-b78d-a46f34af0207 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:53.634055 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389540e-3827-4e32-b78d-a46f34af0207 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:53.638858 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fdb774-7f4d-4404-9b60-b457111b0591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:53.639644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdb774-7f4d-4404-9b60-b457111b0591 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:53.640022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdb774-7f4d-4404-9b60-b457111b0591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:53.640392 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fdb774-7f4d-4404-9b60-b457111b0591 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:53.641231 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdb774-7f4d-4404-9b60-b457111b0591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:53.643129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fdb774-7f4d-4404-9b60-b457111b0591 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:53.653574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1313/2636] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:53.657824 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f38d-32b5-42bf-b85d-17bd0af3d41d admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:53.668177 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab3f38d-32b5-42bf-b85d-17bd0af3d41d admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:53.673028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87851b29-9112-4c99-a953-5d8530c58830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:53.673916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87851b29-9112-4c99-a953-5d8530c58830 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:53.674256 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87851b29-9112-4c99-a953-5d8530c58830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:53.674537 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87851b29-9112-4c99-a953-5d8530c58830 admin admin] PATH_INFO: `/v3/projects/2f657625b1ba4855844201296ce07e47/users/edb8e2c5996a4b3f8caf8da3dabdb8d1/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:53.677699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:53.692194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87851b29-9112-4c99-a953-5d8530c58830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f657625b1ba4855844201296ce07e47, user_id=edb8e2c5996a4b3f8caf8da3dabdb8d1, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:53.694843 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87851b29-9112-4c99-a953-5d8530c58830 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:53.707441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1324/2637] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:53 2022] PUT /identity/v3/projects/2f657625b1ba4855844201296ce07e47/users/edb8e2c5996a4b3f8caf8da3dabdb8d1/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:53.718753 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600b2d3-5772-4dfd-aa78-c7b1a0847122 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:53.719325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600b2d3-5772-4dfd-aa78-c7b1a0847122 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:53.719325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0600b2d3-5772-4dfd-aa78-c7b1a0847122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:53.746997 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-0600b2d3-5772-4dfd-aa78-c7b1a0847122 None None] MFA Rules not processed for user `edb8e2c5996a4b3f8caf8da3dabdb8d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:53.748205 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0600b2d3-5772-4dfd-aa78-c7b1a0847122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:53.814230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1314/2638] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:53.818571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87851b29-9112-4c99-a953-5d8530c58830 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:53.887252 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87851b29-9112-4c99-a953-5d8530c58830 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:53.892326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9563a5-3990-4d51-809e-442154860903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:53.893217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9563a5-3990-4d51-809e-442154860903 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:53.893494 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9563a5-3990-4d51-809e-442154860903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:53.893762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9563a5-3990-4d51-809e-442154860903 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:53.894464 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9563a5-3990-4d51-809e-442154860903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:53.896343 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9563a5-3990-4d51-809e-442154860903 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:53.923144 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1325/2639] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:53 2022] POST /identity/v3/projects => generated 381 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:53.927332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600b2d3-5772-4dfd-aa78-c7b1a0847122 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:53.938650 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0600b2d3-5772-4dfd-aa78-c7b1a0847122 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:53.950103 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:54.001314 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f28568-a06f-4443-aa76-e16c2cec59d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:54.001988 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f28568-a06f-4443-aa76-e16c2cec59d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:54.002266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f28568-a06f-4443-aa76-e16c2cec59d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:54.002539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f28568-a06f-4443-aa76-e16c2cec59d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:54.003236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f28568-a06f-4443-aa76-e16c2cec59d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:54.005198 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f28568-a06f-4443-aa76-e16c2cec59d4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:54.032068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1315/2640] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:53 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:21:54.036125 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9563a5-3990-4d51-809e-442154860903 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:54.047400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9563a5-3990-4d51-809e-442154860903 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:54.053159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370bccc-a83d-4aa6-b03a-d7a3f6596aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:54.053898 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370bccc-a83d-4aa6-b03a-d7a3f6596aba admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:54.054301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370bccc-a83d-4aa6-b03a-d7a3f6596aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:54.054647 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4370bccc-a83d-4aa6-b03a-d7a3f6596aba admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:54.055520 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370bccc-a83d-4aa6-b03a-d7a3f6596aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:54.057313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4370bccc-a83d-4aa6-b03a-d7a3f6596aba admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:54.073895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1326/2641] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:54.079100 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f28568-a06f-4443-aa76-e16c2cec59d4 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:54.089921 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f28568-a06f-4443-aa76-e16c2cec59d4 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:54.094489 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085caa9-9733-4b1d-9280-15c1619b3228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:54.095367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085caa9-9733-4b1d-9280-15c1619b3228 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:54.095687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085caa9-9733-4b1d-9280-15c1619b3228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:54.095909 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085caa9-9733-4b1d-9280-15c1619b3228 admin admin] PATH_INFO: `/v3/projects/4e25e7eb827b4d2aaff4b23835e4264c/users/6f16e76c3a8e4ac895be69ab9c1cd2e6/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:54.113141 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085caa9-9733-4b1d-9280-15c1619b3228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e25e7eb827b4d2aaff4b23835e4264c, user_id=6f16e76c3a8e4ac895be69ab9c1cd2e6, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:54.115775 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6085caa9-9733-4b1d-9280-15c1619b3228 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:54.129254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1316/2642] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:54 2022] PUT /identity/v3/projects/4e25e7eb827b4d2aaff4b23835e4264c/users/6f16e76c3a8e4ac895be69ab9c1cd2e6/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:54.133694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370bccc-a83d-4aa6-b03a-d7a3f6596aba admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:54.144060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4370bccc-a83d-4aa6-b03a-d7a3f6596aba admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:54.200140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b241f43-f5cf-4a02-ab6e-e623add96b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:54.200804 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b241f43-f5cf-4a02-ab6e-e623add96b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:54.201123 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b241f43-f5cf-4a02-ab6e-e623add96b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:54.201395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b241f43-f5cf-4a02-ab6e-e623add96b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:54.202115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b241f43-f5cf-4a02-ab6e-e623add96b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:54.203690 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b241f43-f5cf-4a02-ab6e-e623add96b88 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:54.213642 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1327/2643] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:21:54.217737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085caa9-9733-4b1d-9280-15c1619b3228 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:54.227838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085caa9-9733-4b1d-9280-15c1619b3228 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:54.232477 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4b320-020d-4947-8689-56aaa336fa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:54.233388 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4b320-020d-4947-8689-56aaa336fa42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:54.233661 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4b320-020d-4947-8689-56aaa336fa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:54.233926 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa4b320-020d-4947-8689-56aaa336fa42 admin admin] PATH_INFO: `/v3/projects/4e25e7eb827b4d2aaff4b23835e4264c/users/6f16e76c3a8e4ac895be69ab9c1cd2e6/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:54.236807 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4b320-020d-4947-8689-56aaa336fa42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e25e7eb827b4d2aaff4b23835e4264c, user_id=6f16e76c3a8e4ac895be69ab9c1cd2e6, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:54.239537 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa4b320-020d-4947-8689-56aaa336fa42 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:54.250579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1317/2644] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:54 2022] PUT /identity/v3/projects/4e25e7eb827b4d2aaff4b23835e4264c/users/6f16e76c3a8e4ac895be69ab9c1cd2e6/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:54.261104 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b21314-06e8-46d6-af0b-b23814c887a7 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:54.261394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b21314-06e8-46d6-af0b-b23814c887a7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:54.261672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b21314-06e8-46d6-af0b-b23814c887a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:54.287785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d3b21314-06e8-46d6-af0b-b23814c887a7 None None] MFA Rules not processed for user `6f16e76c3a8e4ac895be69ab9c1cd2e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:54.289013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d3b21314-06e8-46d6-af0b-b23814c887a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:54.332763 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:54.360349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1328/2645] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:54 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:54.371350 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4b320-020d-4947-8689-56aaa336fa42 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:54.373054 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cfa4b320-020d-4947-8689-56aaa336fa42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:54.428577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa4b320-020d-4947-8689-56aaa336fa42 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:54.432230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb69d4e-ad3f-422a-9cdc-a48db6851cfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:54.432865 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb69d4e-ad3f-422a-9cdc-a48db6851cfb service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:54.433202 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb69d4e-ad3f-422a-9cdc-a48db6851cfb service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:54.433475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb69d4e-ad3f-422a-9cdc-a48db6851cfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:54.442581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb69d4e-ad3f-422a-9cdc-a48db6851cfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:54.443995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb69d4e-ad3f-422a-9cdc-a48db6851cfb service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:54.453840 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1318/2646] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:54 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:54.547545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b21314-06e8-46d6-af0b-b23814c887a7 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:54.557555 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b21314-06e8-46d6-af0b-b23814c887a7 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:54.561099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fe1b4-f332-46b0-96cc-0f6f1cd892ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:54.561733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fe1b4-f332-46b0-96cc-0f6f1cd892ba service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:54.562014 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fe1b4-f332-46b0-96cc-0f6f1cd892ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:54.562286 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481fe1b4-f332-46b0-96cc-0f6f1cd892ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:54.618591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fe1b4-f332-46b0-96cc-0f6f1cd892ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:54.620559 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481fe1b4-f332-46b0-96cc-0f6f1cd892ba service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:54.629593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1329/2647] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:55.311082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb69d4e-ad3f-422a-9cdc-a48db6851cfb service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:55.312494 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-2cb69d4e-ad3f-422a-9cdc-a48db6851cfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:55.402446 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:55.422708 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb69d4e-ad3f-422a-9cdc-a48db6851cfb service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:55.433230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d5fc74-46a7-409d-9237-a7a7564ce5fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff4c42a4f4f549378e5bc1cd92f801c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:55.433230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5fc74-46a7-409d-9237-a7a7564ce5fa service placement] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:55.433230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5fc74-46a7-409d-9237-a7a7564ce5fa service placement] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:55.433230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d5fc74-46a7-409d-9237-a7a7564ce5fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:55.433598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f8d5fc74-46a7-409d-9237-a7a7564ce5fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:55.444058 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5fc74-46a7-409d-9237-a7a7564ce5fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:55.445675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d5fc74-46a7-409d-9237-a7a7564ce5fa service placement] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:55.455932 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1319/2648] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:57.201703 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fe1b4-f332-46b0-96cc-0f6f1cd892ba service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:57.273379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481fe1b4-f332-46b0-96cc-0f6f1cd892ba service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:57.278526 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029c5a3-382e-4b8e-b01e-debb591892ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:57.279482 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029c5a3-382e-4b8e-b01e-debb591892ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:57.279830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029c5a3-382e-4b8e-b01e-debb591892ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:57.280169 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a029c5a3-382e-4b8e-b01e-debb591892ff admin admin] PATH_INFO: `/v3/users/edb8e2c5996a4b3f8caf8da3dabdb8d1` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:57.281449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029c5a3-382e-4b8e-b01e-debb591892ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=edb8e2c5996a4b3f8caf8da3dabdb8d1)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:57.283419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a029c5a3-382e-4b8e-b01e-debb591892ff admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:57.340385 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a029c5a3-382e-4b8e-b01e-debb591892ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edb8e2c5996a4b3f8caf8da3dabdb8d1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:57.341098 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a029c5a3-382e-4b8e-b01e-debb591892ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edb8e2c5996a4b3f8caf8da3dabdb8d1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:57.352778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a029c5a3-382e-4b8e-b01e-debb591892ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edb8e2c5996a4b3f8caf8da3dabdb8d1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:57.359631 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a029c5a3-382e-4b8e-b01e-debb591892ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edb8e2c5996a4b3f8caf8da3dabdb8d1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:57.366920 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-a029c5a3-382e-4b8e-b01e-debb591892ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edb8e2c5996a4b3f8caf8da3dabdb8d1'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:57.368453 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:57.376364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1330/2649] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:57 2022] DELETE /identity/v3/users/edb8e2c5996a4b3f8caf8da3dabdb8d1 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:21:57.380950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2564e-ec91-4eb0-80fd-704aa1ca3e9b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:57.381435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2564e-ec91-4eb0-80fd-704aa1ca3e9b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:57.381551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f2564e-ec91-4eb0-80fd-704aa1ca3e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:57.400152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-55f2564e-ec91-4eb0-80fd-704aa1ca3e9b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:57.401951 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-55f2564e-ec91-4eb0-80fd-704aa1ca3e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:57.458826 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1320/2650] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:57.469081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029c5a3-382e-4b8e-b01e-debb591892ff admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:57.470561 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-a029c5a3-382e-4b8e-b01e-debb591892ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:57.537949 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a029c5a3-382e-4b8e-b01e-debb591892ff admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:57.541631 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc1885-fe6a-4967-8500-242f5ecd5281 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:57.542495 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc1885-fe6a-4967-8500-242f5ecd5281 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:57.542661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc1885-fe6a-4967-8500-242f5ecd5281 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:57.542940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc1885-fe6a-4967-8500-242f5ecd5281 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:57.552505 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc1885-fe6a-4967-8500-242f5ecd5281 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:57.553956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dc1885-fe6a-4967-8500-242f5ecd5281 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:57.564187 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1331/2651] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:21:57.836415 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2564e-ec91-4eb0-80fd-704aa1ca3e9b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:57.843492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-55f2564e-ec91-4eb0-80fd-704aa1ca3e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:57.857781 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f2564e-ec91-4eb0-80fd-704aa1ca3e9b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:57.863057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b4a621-4d5e-4c5e-8844-32dea012073f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:57.864245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4a621-4d5e-4c5e-8844-32dea012073f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:57.864551 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4a621-4d5e-4c5e-8844-32dea012073f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:57.864827 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b4a621-4d5e-4c5e-8844-32dea012073f admin admin] PATH_INFO: `/v3/projects/2f657625b1ba4855844201296ce07e47` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:57.866078 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4a621-4d5e-4c5e-8844-32dea012073f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f657625b1ba4855844201296ce07e47)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:57.867959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b4a621-4d5e-4c5e-8844-32dea012073f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:57.921749 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-84b4a621-4d5e-4c5e-8844-32dea012073f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f657625b1ba4855844201296ce07e47', 'request_id': 'req-84b4a621-4d5e-4c5e-8844-32dea012073f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:57.927893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-84b4a621-4d5e-4c5e-8844-32dea012073f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f657625b1ba4855844201296ce07e47', 'request_id': 'req-84b4a621-4d5e-4c5e-8844-32dea012073f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:57.935949 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-84b4a621-4d5e-4c5e-8844-32dea012073f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f657625b1ba4855844201296ce07e47', 'request_id': 'req-84b4a621-4d5e-4c5e-8844-32dea012073f'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:57.946876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-84b4a621-4d5e-4c5e-8844-32dea012073f admin admin] The token cache is being invalidate because project 2f657625b1ba4855844201296ce07e47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:57.947163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-84b4a621-4d5e-4c5e-8844-32dea012073f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:57.948401 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-84b4a621-4d5e-4c5e-8844-32dea012073f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:57.950050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1321/2652] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:57 2022] DELETE /identity/v3/projects/2f657625b1ba4855844201296ce07e47 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:57.954136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc1885-fe6a-4967-8500-242f5ecd5281 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:57.966201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc1885-fe6a-4967-8500-242f5ecd5281 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:58.022982 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85a512-f2dc-4337-9dc5-85ff695a0c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:58.023860 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85a512-f2dc-4337-9dc5-85ff695a0c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:58.024134 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85a512-f2dc-4337-9dc5-85ff695a0c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:58.024402 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f85a512-f2dc-4337-9dc5-85ff695a0c7a admin admin] PATH_INFO: `/v3/users/6f16e76c3a8e4ac895be69ab9c1cd2e6` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:58.025673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85a512-f2dc-4337-9dc5-85ff695a0c7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f16e76c3a8e4ac895be69ab9c1cd2e6)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:58.028216 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f85a512-f2dc-4337-9dc5-85ff695a0c7a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:58.093236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41c6cb-7ca8-4eb3-9f50-fb126ac4d9a3 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:58.093630 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41c6cb-7ca8-4eb3-9f50-fb126ac4d9a3 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:58.093981 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41c6cb-7ca8-4eb3-9f50-fb126ac4d9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:58.099300 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6f85a512-f2dc-4337-9dc5-85ff695a0c7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f16e76c3a8e4ac895be69ab9c1cd2e6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:58.100442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6f85a512-f2dc-4337-9dc5-85ff695a0c7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f16e76c3a8e4ac895be69ab9c1cd2e6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:58.115851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6f85a512-f2dc-4337-9dc5-85ff695a0c7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f16e76c3a8e4ac895be69ab9c1cd2e6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:58.123225 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6f85a512-f2dc-4337-9dc5-85ff695a0c7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f16e76c3a8e4ac895be69ab9c1cd2e6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:58.126731 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ec41c6cb-7ca8-4eb3-9f50-fb126ac4d9a3 None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:58.127979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ec41c6cb-7ca8-4eb3-9f50-fb126ac4d9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:58.132641 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-6f85a512-f2dc-4337-9dc5-85ff695a0c7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f16e76c3a8e4ac895be69ab9c1cd2e6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:58.144277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1332/2653] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:21:57 2022] DELETE /identity/v3/users/6f16e76c3a8e4ac895be69ab9c1cd2e6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:58.198491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:58.223421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1322/2654] 10.222.0.61 () {58 vars in 967 bytes} [Thu Jul 14 10:21:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:58.238637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85a512-f2dc-4337-9dc5-85ff695a0c7a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:58.241530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6f85a512-f2dc-4337-9dc5-85ff695a0c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:58.271045 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:58.328353 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f85a512-f2dc-4337-9dc5-85ff695a0c7a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:58.334667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930d4e3-3f81-49b1-8ff4-f5378db302f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:58.335473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930d4e3-3f81-49b1-8ff4-f5378db302f4 service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:58.335757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930d4e3-3f81-49b1-8ff4-f5378db302f4 service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:58.336033 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930d4e3-3f81-49b1-8ff4-f5378db302f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:58.418033 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930d4e3-3f81-49b1-8ff4-f5378db302f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:58.419636 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930d4e3-3f81-49b1-8ff4-f5378db302f4 service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:58.429819 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1333/2655] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:21:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:21:58.536541 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41c6cb-7ca8-4eb3-9f50-fb126ac4d9a3 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:58.538188 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ec41c6cb-7ca8-4eb3-9f50-fb126ac4d9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:58.557429 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41c6cb-7ca8-4eb3-9f50-fb126ac4d9a3 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:58.627667 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b70248-70c8-4bb0-83e5-a3f8fed60549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:58.628747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b70248-70c8-4bb0-83e5-a3f8fed60549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:58.629163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b70248-70c8-4bb0-83e5-a3f8fed60549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:58.629579 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b70248-70c8-4bb0-83e5-a3f8fed60549 admin admin] PATH_INFO: `/v3/projects/4e25e7eb827b4d2aaff4b23835e4264c` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:58.631700 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b70248-70c8-4bb0-83e5-a3f8fed60549 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e25e7eb827b4d2aaff4b23835e4264c)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:58.633928 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b70248-70c8-4bb0-83e5-a3f8fed60549 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:58.708718 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-20b70248-70c8-4bb0-83e5-a3f8fed60549 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e25e7eb827b4d2aaff4b23835e4264c', 'request_id': 'req-20b70248-70c8-4bb0-83e5-a3f8fed60549'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:58.719491 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-20b70248-70c8-4bb0-83e5-a3f8fed60549 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e25e7eb827b4d2aaff4b23835e4264c', 'request_id': 'req-20b70248-70c8-4bb0-83e5-a3f8fed60549'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:58.732173 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-20b70248-70c8-4bb0-83e5-a3f8fed60549 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e25e7eb827b4d2aaff4b23835e4264c', 'request_id': 'req-20b70248-70c8-4bb0-83e5-a3f8fed60549'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:58.751544 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-20b70248-70c8-4bb0-83e5-a3f8fed60549 admin admin] The token cache is being invalidate because project 4e25e7eb827b4d2aaff4b23835e4264c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:21:58.751959 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-20b70248-70c8-4bb0-83e5-a3f8fed60549 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:58.752876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-20b70248-70c8-4bb0-83e5-a3f8fed60549 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:21:58.755570 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1323/2656] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:21:58 2022] DELETE /identity/v3/projects/4e25e7eb827b4d2aaff4b23835e4264c => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:58.761800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56040605-a1f8-4001-92ca-10a68d44f184 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:58.762269 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56040605-a1f8-4001-92ca-10a68d44f184 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:58.762701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56040605-a1f8-4001-92ca-10a68d44f184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:58.793717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-56040605-a1f8-4001-92ca-10a68d44f184 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:58.795138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-56040605-a1f8-4001-92ca-10a68d44f184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:58.888825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1334/2657] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:58.896950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71506d9-50ff-44ca-bcf1-ddfc3eec1d42 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:58.897246 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71506d9-50ff-44ca-bcf1-ddfc3eec1d42 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:58.897525 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71506d9-50ff-44ca-bcf1-ddfc3eec1d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:58.916477 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-c71506d9-50ff-44ca-bcf1-ddfc3eec1d42 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:58.917844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-c71506d9-50ff-44ca-bcf1-ddfc3eec1d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:58.928376 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1324/2658] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:58.933419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56040605-a1f8-4001-92ca-10a68d44f184 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:58.944702 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56040605-a1f8-4001-92ca-10a68d44f184 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:58.952526 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca1d35-79d6-46f4-945f-1e6a538bc064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:58.953721 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca1d35-79d6-46f4-945f-1e6a538bc064 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:58.954008 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca1d35-79d6-46f4-945f-1e6a538bc064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:58.954313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca1d35-79d6-46f4-945f-1e6a538bc064 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:58.955435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca1d35-79d6-46f4-945f-1e6a538bc064 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:21:58.955956 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca1d35-79d6-46f4-945f-1e6a538bc064 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:58.957882 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ca1d35-79d6-46f4-945f-1e6a538bc064 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:58.971024 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1335/2659] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:21:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:21:58.975723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71506d9-50ff-44ca-bcf1-ddfc3eec1d42 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:58.985915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71506d9-50ff-44ca-bcf1-ddfc3eec1d42 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:58.990789 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee24a18-30bf-4009-9c72-486110cef00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:58.991557 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee24a18-30bf-4009-9c72-486110cef00f admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:58.991903 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee24a18-30bf-4009-9c72-486110cef00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:58.992304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee24a18-30bf-4009-9c72-486110cef00f admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:58.993029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee24a18-30bf-4009-9c72-486110cef00f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:58.994940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee24a18-30bf-4009-9c72-486110cef00f admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:59.020846 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1325/2660] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:58 2022] POST /identity/v3/projects => generated 407 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:21:59.025090 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca1d35-79d6-46f4-945f-1e6a538bc064 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:59.036128 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ca1d35-79d6-46f4-945f-1e6a538bc064 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:59.090286 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:59.111349 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a216f-c53c-456c-a13e-2424afbae722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:59.111972 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a216f-c53c-456c-a13e-2424afbae722 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:59.112231 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a216f-c53c-456c-a13e-2424afbae722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:59.112506 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a216f-c53c-456c-a13e-2424afbae722 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:59.113727 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a216f-c53c-456c-a13e-2424afbae722 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:59.115244 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a216f-c53c-456c-a13e-2424afbae722 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:59.138135 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1336/2661] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:59 2022] POST /identity/v3/users => generated 304 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:59.142512 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee24a18-30bf-4009-9c72-486110cef00f admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:59.153310 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee24a18-30bf-4009-9c72-486110cef00f admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:59.158848 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505c86b-2b1a-407a-bda0-76433bcf8335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:59.159553 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505c86b-2b1a-407a-bda0-76433bcf8335 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:59.159906 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505c86b-2b1a-407a-bda0-76433bcf8335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:59.160262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505c86b-2b1a-407a-bda0-76433bcf8335 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:59.161112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505c86b-2b1a-407a-bda0-76433bcf8335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:59.163067 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6505c86b-2b1a-407a-bda0-76433bcf8335 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:59.173991 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1326/2662] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:59.178173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a216f-c53c-456c-a13e-2424afbae722 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:59.195684 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a216f-c53c-456c-a13e-2424afbae722 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:59.202255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc78eea-8e1c-4530-87cd-87f0822ab9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:59.203030 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc78eea-8e1c-4530-87cd-87f0822ab9db admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:59.203312 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc78eea-8e1c-4530-87cd-87f0822ab9db admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:59.203592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc78eea-8e1c-4530-87cd-87f0822ab9db admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:59.204320 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc78eea-8e1c-4530-87cd-87f0822ab9db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:59.205940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc78eea-8e1c-4530-87cd-87f0822ab9db admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:59.217357 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1337/2663] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:59.221877 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505c86b-2b1a-407a-bda0-76433bcf8335 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:59.232478 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505c86b-2b1a-407a-bda0-76433bcf8335 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:59.243064 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed3507-2016-4a11-bb2b-90443695af08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:59.243970 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed3507-2016-4a11-bb2b-90443695af08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:59.244815 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed3507-2016-4a11-bb2b-90443695af08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:59.245213 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed3507-2016-4a11-bb2b-90443695af08 admin admin] PATH_INFO: `/v3/projects/0cd3780fb7a24fc3bc4c018f7dbc1f39/users/2a06951cf07d47a1b4419daacdbd5078/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:59.260799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed3507-2016-4a11-bb2b-90443695af08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cd3780fb7a24fc3bc4c018f7dbc1f39, user_id=2a06951cf07d47a1b4419daacdbd5078, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:59.263455 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed3507-2016-4a11-bb2b-90443695af08 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:59.276327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1327/2664] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:59 2022] PUT /identity/v3/projects/0cd3780fb7a24fc3bc4c018f7dbc1f39/users/2a06951cf07d47a1b4419daacdbd5078/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:59.340474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf2640-8e95-4984-b515-f817bd35e2d7 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:59.341067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf2640-8e95-4984-b515-f817bd35e2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:59.341168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf2640-8e95-4984-b515-f817bd35e2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:59.370244 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-96cf2640-8e95-4984-b515-f817bd35e2d7 None None] MFA Rules not processed for user `2a06951cf07d47a1b4419daacdbd5078`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:59.371379 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-96cf2640-8e95-4984-b515-f817bd35e2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:59.433986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1338/2665] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:59 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:21:59.438543 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed3507-2016-4a11-bb2b-90443695af08 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:59.459460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:21:59.501724 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed3507-2016-4a11-bb2b-90443695af08 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:59.507751 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af6a02-6aea-4f38-86c4-572c5b70c0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:59.508407 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af6a02-6aea-4f38-86c4-572c5b70c0ba admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:59.508685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af6a02-6aea-4f38-86c4-572c5b70c0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:59.509045 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af6a02-6aea-4f38-86c4-572c5b70c0ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:59.509761 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af6a02-6aea-4f38-86c4-572c5b70c0ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:59.511960 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af6a02-6aea-4f38-86c4-572c5b70c0ba admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:59.542066 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1328/2666] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:21:59 2022] POST /identity/v3/projects => generated 407 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:59.544174 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf2640-8e95-4984-b515-f817bd35e2d7 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:59.555100 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf2640-8e95-4984-b515-f817bd35e2d7 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:59.613527 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092d84c-565c-4a29-8a45-2e6e13517cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:59.614191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092d84c-565c-4a29-8a45-2e6e13517cdc admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:59.614470 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092d84c-565c-4a29-8a45-2e6e13517cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:59.614739 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c092d84c-565c-4a29-8a45-2e6e13517cdc admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:59.615433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092d84c-565c-4a29-8a45-2e6e13517cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:59.617395 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c092d84c-565c-4a29-8a45-2e6e13517cdc admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:59.638236 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1339/2667] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:21:59 2022] POST /identity/v3/users => generated 310 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:21:59.642440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af6a02-6aea-4f38-86c4-572c5b70c0ba admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:59.653018 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af6a02-6aea-4f38-86c4-572c5b70c0ba admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:59.657606 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6017454-2f0d-4777-82b4-2350a54a0d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:59.658269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6017454-2f0d-4777-82b4-2350a54a0d09 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:59.658672 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6017454-2f0d-4777-82b4-2350a54a0d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:59.658810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6017454-2f0d-4777-82b4-2350a54a0d09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:59.659527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6017454-2f0d-4777-82b4-2350a54a0d09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:59.661137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6017454-2f0d-4777-82b4-2350a54a0d09 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:59.672403 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1329/2668] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:59.677270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092d84c-565c-4a29-8a45-2e6e13517cdc admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:59.688119 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c092d84c-565c-4a29-8a45-2e6e13517cdc admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:59.692713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4cef57-349f-4d59-a900-8917a13eadd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:59.693631 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4cef57-349f-4d59-a900-8917a13eadd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:59.693914 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4cef57-349f-4d59-a900-8917a13eadd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:59.694186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4cef57-349f-4d59-a900-8917a13eadd6 admin admin] PATH_INFO: `/v3/projects/33aa7ee47aa84200b21799465e18f676/users/57e40420ba5349fba285d586ed1d8e17/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:59.708890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4cef57-349f-4d59-a900-8917a13eadd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33aa7ee47aa84200b21799465e18f676, user_id=57e40420ba5349fba285d586ed1d8e17, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:59.711682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4cef57-349f-4d59-a900-8917a13eadd6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:59.724473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1340/2669] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:59 2022] PUT /identity/v3/projects/33aa7ee47aa84200b21799465e18f676/users/57e40420ba5349fba285d586ed1d8e17/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:59.728424 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6017454-2f0d-4777-82b4-2350a54a0d09 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:59.738710 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6017454-2f0d-4777-82b4-2350a54a0d09 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:59.793137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65877e4c-ff63-42cc-87a0-b3b13e3b776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:59.793784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65877e4c-ff63-42cc-87a0-b3b13e3b776b admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:59.794087 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65877e4c-ff63-42cc-87a0-b3b13e3b776b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:59.794356 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65877e4c-ff63-42cc-87a0-b3b13e3b776b admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:59.795097 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65877e4c-ff63-42cc-87a0-b3b13e3b776b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:59.796661 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65877e4c-ff63-42cc-87a0-b3b13e3b776b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:59.806010 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1330/2670] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:21:59.810717 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4cef57-349f-4d59-a900-8917a13eadd6 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:59.820661 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4cef57-349f-4d59-a900-8917a13eadd6 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:21:59.825173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c73534-2652-4e0f-ada6-432a4ec69fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:21:59.826038 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c73534-2652-4e0f-ada6-432a4ec69fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:59.826310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c73534-2652-4e0f-ada6-432a4ec69fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:59.826583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c73534-2652-4e0f-ada6-432a4ec69fdd admin admin] PATH_INFO: `/v3/projects/33aa7ee47aa84200b21799465e18f676/users/57e40420ba5349fba285d586ed1d8e17/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:59.828563 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c73534-2652-4e0f-ada6-432a4ec69fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=33aa7ee47aa84200b21799465e18f676, user_id=57e40420ba5349fba285d586ed1d8e17, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:21:59.831162 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c73534-2652-4e0f-ada6-432a4ec69fdd admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:21:59.841534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1341/2671] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:21:59 2022] PUT /identity/v3/projects/33aa7ee47aa84200b21799465e18f676/users/57e40420ba5349fba285d586ed1d8e17/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:21:59.851439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e346a3-98db-48d8-b14f-cfeabfd5a699 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:21:59.851726 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e346a3-98db-48d8-b14f-cfeabfd5a699 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:21:59.851999 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e346a3-98db-48d8-b14f-cfeabfd5a699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:21:59.878147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-88e346a3-98db-48d8-b14f-cfeabfd5a699 None None] MFA Rules not processed for user `57e40420ba5349fba285d586ed1d8e17`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:21:59.879368 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-88e346a3-98db-48d8-b14f-cfeabfd5a699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:59.945147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1331/2672] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:21:59 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:21:59.955837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c73534-2652-4e0f-ada6-432a4ec69fdd admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:21:59.957556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-90c73534-2652-4e0f-ada6-432a4ec69fdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:21:59.968526 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:00.017299 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c73534-2652-4e0f-ada6-432a4ec69fdd admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:00.021592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8536a46-b6f8-4b80-8ec7-ef503089813e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:00.022239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8536a46-b6f8-4b80-8ec7-ef503089813e service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:00.022519 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8536a46-b6f8-4b80-8ec7-ef503089813e service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:00.022789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8536a46-b6f8-4b80-8ec7-ef503089813e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:00.033752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8536a46-b6f8-4b80-8ec7-ef503089813e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:00.035330 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8536a46-b6f8-4b80-8ec7-ef503089813e service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:00.045316 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1342/2673] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:21:59 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:22:00.193874 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e346a3-98db-48d8-b14f-cfeabfd5a699 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:00.206175 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e346a3-98db-48d8-b14f-cfeabfd5a699 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:00.209782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d080c-c436-4a19-b38e-f75f6a719d7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:00.210428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d080c-c436-4a19-b38e-f75f6a719d7c service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:00.210709 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d080c-c436-4a19-b38e-f75f6a719d7c service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:00.210984 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d080c-c436-4a19-b38e-f75f6a719d7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:00.223606 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:00.272656 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d080c-c436-4a19-b38e-f75f6a719d7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:00.274103 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d080c-c436-4a19-b38e-f75f6a719d7c service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:00.283900 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1332/2674] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:22:00 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:22:02.285255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8536a46-b6f8-4b80-8ec7-ef503089813e service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:02.286637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d8536a46-b6f8-4b80-8ec7-ef503089813e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:02.348600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8536a46-b6f8-4b80-8ec7-ef503089813e service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:02.353084 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d6084-dd57-4c83-88c0-d64e43b1e72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:02.353970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d6084-dd57-4c83-88c0-d64e43b1e72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:02.354248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d6084-dd57-4c83-88c0-d64e43b1e72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:02.354517 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599d6084-dd57-4c83-88c0-d64e43b1e72a admin admin] PATH_INFO: `/v3/users/eb3798d83a6644daa6f42f8456466cea` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:02.355673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d6084-dd57-4c83-88c0-d64e43b1e72a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb3798d83a6644daa6f42f8456466cea)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:02.357596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599d6084-dd57-4c83-88c0-d64e43b1e72a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:02.402760 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-599d6084-dd57-4c83-88c0-d64e43b1e72a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb3798d83a6644daa6f42f8456466cea'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:02.403404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-599d6084-dd57-4c83-88c0-d64e43b1e72a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb3798d83a6644daa6f42f8456466cea'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:02.414920 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-599d6084-dd57-4c83-88c0-d64e43b1e72a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb3798d83a6644daa6f42f8456466cea'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:02.422181 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-599d6084-dd57-4c83-88c0-d64e43b1e72a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb3798d83a6644daa6f42f8456466cea'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:02.429261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-599d6084-dd57-4c83-88c0-d64e43b1e72a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb3798d83a6644daa6f42f8456466cea'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:02.435733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1343/2675] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:22:02 2022] DELETE /identity/v3/users/eb3798d83a6644daa6f42f8456466cea => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:22:02.440469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e846152-62e3-4cdb-a59e-38339cbd62d6 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:02.440966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e846152-62e3-4cdb-a59e-38339cbd62d6 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:02.441222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e846152-62e3-4cdb-a59e-38339cbd62d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:02.459800 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4e846152-62e3-4cdb-a59e-38339cbd62d6 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:02.461022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4e846152-62e3-4cdb-a59e-38339cbd62d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:02.517995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1333/2676] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:22:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:22:02.528142 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d6084-dd57-4c83-88c0-d64e43b1e72a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:02.529573 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-599d6084-dd57-4c83-88c0-d64e43b1e72a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:02.564374 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:02.610242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599d6084-dd57-4c83-88c0-d64e43b1e72a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:02.613903 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c599c3-1e67-4d46-bdc5-027eefcaf0cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:02.614542 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c599c3-1e67-4d46-bdc5-027eefcaf0cc service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:02.614822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c599c3-1e67-4d46-bdc5-027eefcaf0cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:02.615096 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c599c3-1e67-4d46-bdc5-027eefcaf0cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:02.624032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c599c3-1e67-4d46-bdc5-027eefcaf0cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:02.625493 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c599c3-1e67-4d46-bdc5-027eefcaf0cc service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:02.634746 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1344/2677] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:22:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:22:02.896942 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e846152-62e3-4cdb-a59e-38339cbd62d6 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:02.898509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4e846152-62e3-4cdb-a59e-38339cbd62d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:02.910816 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e846152-62e3-4cdb-a59e-38339cbd62d6 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:02.915718 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f49f22-a9fd-45ef-8ec9-595c666c0903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:02.917257 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f49f22-a9fd-45ef-8ec9-595c666c0903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:02.917661 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f49f22-a9fd-45ef-8ec9-595c666c0903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:02.917845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f49f22-a9fd-45ef-8ec9-595c666c0903 admin admin] PATH_INFO: `/v3/projects/bf1ba07f5cc949d7b269cf95f48a17f6` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:02.919082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f49f22-a9fd-45ef-8ec9-595c666c0903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf1ba07f5cc949d7b269cf95f48a17f6)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:02.921016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f49f22-a9fd-45ef-8ec9-595c666c0903 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:02.967205 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b2f49f22-a9fd-45ef-8ec9-595c666c0903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf1ba07f5cc949d7b269cf95f48a17f6', 'request_id': 'req-b2f49f22-a9fd-45ef-8ec9-595c666c0903'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:02.973498 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b2f49f22-a9fd-45ef-8ec9-595c666c0903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf1ba07f5cc949d7b269cf95f48a17f6', 'request_id': 'req-b2f49f22-a9fd-45ef-8ec9-595c666c0903'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:02.981453 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b2f49f22-a9fd-45ef-8ec9-595c666c0903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf1ba07f5cc949d7b269cf95f48a17f6', 'request_id': 'req-b2f49f22-a9fd-45ef-8ec9-595c666c0903'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:02.992037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b2f49f22-a9fd-45ef-8ec9-595c666c0903 admin admin] The token cache is being invalidate because project bf1ba07f5cc949d7b269cf95f48a17f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:22:02.992318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b2f49f22-a9fd-45ef-8ec9-595c666c0903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:02.992986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b2f49f22-a9fd-45ef-8ec9-595c666c0903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:02.994608 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1334/2678] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:22:02 2022] DELETE /identity/v3/projects/bf1ba07f5cc949d7b269cf95f48a17f6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:22:02.999586 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6171e-2ba7-4a7a-a769-386f8109bed5 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:02.999876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6171e-2ba7-4a7a-a769-386f8109bed5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:03.000184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6171e-2ba7-4a7a-a769-386f8109bed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:03.018502 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5fc6171e-2ba7-4a7a-a769-386f8109bed5 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:03.019701 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5fc6171e-2ba7-4a7a-a769-386f8109bed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:03.076459 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1345/2679] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:22:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:22:03.084343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c80fe9-1f9b-4d76-ade3-d79d1e12050e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:03.084629 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c80fe9-1f9b-4d76-ade3-d79d1e12050e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:03.084901 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c80fe9-1f9b-4d76-ade3-d79d1e12050e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:03.105710 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f6c80fe9-1f9b-4d76-ade3-d79d1e12050e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:03.106908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f6c80fe9-1f9b-4d76-ade3-d79d1e12050e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:03.114813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1335/2680] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:22:03.118916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6171e-2ba7-4a7a-a769-386f8109bed5 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:03.128380 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6171e-2ba7-4a7a-a769-386f8109bed5 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:03.132803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815e254-97b1-42b2-8d97-2d394005168a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:03.133485 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815e254-97b1-42b2-8d97-2d394005168a admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:03.133761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815e254-97b1-42b2-8d97-2d394005168a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:03.134028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e815e254-97b1-42b2-8d97-2d394005168a admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:03.134740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815e254-97b1-42b2-8d97-2d394005168a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:22:03.135061 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815e254-97b1-42b2-8d97-2d394005168a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:03.136623 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e815e254-97b1-42b2-8d97-2d394005168a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:03.148159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1346/2681] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:22:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:22:03.152689 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c80fe9-1f9b-4d76-ade3-d79d1e12050e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:03.161752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c80fe9-1f9b-4d76-ade3-d79d1e12050e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:03.167310 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd594b8-c8e2-41c4-8adb-22cfddb963ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:03.167962 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd594b8-c8e2-41c4-8adb-22cfddb963ac admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:03.168242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd594b8-c8e2-41c4-8adb-22cfddb963ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:03.168541 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd594b8-c8e2-41c4-8adb-22cfddb963ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:03.169272 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd594b8-c8e2-41c4-8adb-22cfddb963ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:03.171193 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd594b8-c8e2-41c4-8adb-22cfddb963ac admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:03.196838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1336/2682] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:22:03 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:22:03.201447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815e254-97b1-42b2-8d97-2d394005168a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:03.211233 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e815e254-97b1-42b2-8d97-2d394005168a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:03.267158 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6f145-2297-492d-9492-d5e0014a3d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:03.267827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6f145-2297-492d-9492-d5e0014a3d07 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:03.268108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6f145-2297-492d-9492-d5e0014a3d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:03.268381 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da6f145-2297-492d-9492-d5e0014a3d07 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:03.269124 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6f145-2297-492d-9492-d5e0014a3d07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:03.271026 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da6f145-2297-492d-9492-d5e0014a3d07 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:03.290891 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1347/2683] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:22:03 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:22:03.295490 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd594b8-c8e2-41c4-8adb-22cfddb963ac admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:03.305696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd594b8-c8e2-41c4-8adb-22cfddb963ac admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:03.310310 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e494d18-7e18-4023-845b-5cef21b10889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:03.310997 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e494d18-7e18-4023-845b-5cef21b10889 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:03.311274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e494d18-7e18-4023-845b-5cef21b10889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:03.311541 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e494d18-7e18-4023-845b-5cef21b10889 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:03.312262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e494d18-7e18-4023-845b-5cef21b10889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:03.313872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e494d18-7e18-4023-845b-5cef21b10889 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:03.315866 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:03.326161 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1337/2684] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:22:03.329777 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6f145-2297-492d-9492-d5e0014a3d07 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:03.340094 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da6f145-2297-492d-9492-d5e0014a3d07 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:03.344622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f23d4e1-53a1-4683-88fe-399bec3218f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:03.345311 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23d4e1-53a1-4683-88fe-399bec3218f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:03.345587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23d4e1-53a1-4683-88fe-399bec3218f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:03.345857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f23d4e1-53a1-4683-88fe-399bec3218f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:03.346580 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23d4e1-53a1-4683-88fe-399bec3218f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:03.348140 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f23d4e1-53a1-4683-88fe-399bec3218f0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:03.357780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1348/2685] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:22:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:22:03.361777 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e494d18-7e18-4023-845b-5cef21b10889 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:03.371378 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e494d18-7e18-4023-845b-5cef21b10889 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:03.375930 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d59e2-ee90-480a-8164-e1d89091fb77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:03.376793 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d59e2-ee90-480a-8164-e1d89091fb77 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:03.377108 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d59e2-ee90-480a-8164-e1d89091fb77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:03.377380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d59e2-ee90-480a-8164-e1d89091fb77 admin admin] PATH_INFO: `/v3/projects/936db5c6e71f4012a4838d484c1ca92e/users/1c009a915aad4fe9b8c334e64ecb2e48/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:03.392527 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d59e2-ee90-480a-8164-e1d89091fb77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=936db5c6e71f4012a4838d484c1ca92e, user_id=1c009a915aad4fe9b8c334e64ecb2e48, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:03.395266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d59e2-ee90-480a-8164-e1d89091fb77 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:03.408237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1338/2686] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:22:03 2022] PUT /identity/v3/projects/936db5c6e71f4012a4838d484c1ca92e/users/1c009a915aad4fe9b8c334e64ecb2e48/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:22:03.413537 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc51820-2c89-4e95-a03a-cde885a6dbe8 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:03.413822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc51820-2c89-4e95-a03a-cde885a6dbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:03.414093 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc51820-2c89-4e95-a03a-cde885a6dbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:03.431113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-8fc51820-2c89-4e95-a03a-cde885a6dbe8 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:03.432326 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8fc51820-2c89-4e95-a03a-cde885a6dbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:03.444593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:03.490511 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1349/2687] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:22:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:22:03.500017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d59e2-ee90-480a-8164-e1d89091fb77 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:03.501712 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e15d59e2-ee90-480a-8164-e1d89091fb77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:03.558940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d59e2-ee90-480a-8164-e1d89091fb77 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:03.562428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0558d7e-db37-4d9e-9ba7-b8c6106c9b35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:03.563063 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0558d7e-db37-4d9e-9ba7-b8c6106c9b35 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:03.563341 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0558d7e-db37-4d9e-9ba7-b8c6106c9b35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:03.563617 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0558d7e-db37-4d9e-9ba7-b8c6106c9b35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:03.573879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0558d7e-db37-4d9e-9ba7-b8c6106c9b35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:03.575361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0558d7e-db37-4d9e-9ba7-b8c6106c9b35 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:03.585472 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1339/2688] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:22:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:22:07.704747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc51820-2c89-4e95-a03a-cde885a6dbe8 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:07.706410 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8fc51820-2c89-4e95-a03a-cde885a6dbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:07.724492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc51820-2c89-4e95-a03a-cde885a6dbe8 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:07.729423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d28ac8-7450-478a-84ec-02f7ccb274bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:07.730422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d28ac8-7450-478a-84ec-02f7ccb274bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:07.730797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d28ac8-7450-478a-84ec-02f7ccb274bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:07.731164 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d28ac8-7450-478a-84ec-02f7ccb274bf admin admin] PATH_INFO: `/v3/users/2a06951cf07d47a1b4419daacdbd5078` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:07.732457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d28ac8-7450-478a-84ec-02f7ccb274bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a06951cf07d47a1b4419daacdbd5078)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:07.734653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d28ac8-7450-478a-84ec-02f7ccb274bf admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:07.811068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-72d28ac8-7450-478a-84ec-02f7ccb274bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a06951cf07d47a1b4419daacdbd5078'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:07.811778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-72d28ac8-7450-478a-84ec-02f7ccb274bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a06951cf07d47a1b4419daacdbd5078'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:07.827816 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-72d28ac8-7450-478a-84ec-02f7ccb274bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a06951cf07d47a1b4419daacdbd5078'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:07.835446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-72d28ac8-7450-478a-84ec-02f7ccb274bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a06951cf07d47a1b4419daacdbd5078'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:07.846272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-72d28ac8-7450-478a-84ec-02f7ccb274bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a06951cf07d47a1b4419daacdbd5078'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:07.846910 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4c799-44ed-4256-9e7b-ea981d5c9926 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:07.847742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4c799-44ed-4256-9e7b-ea981d5c9926 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:07.848173 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d4c799-44ed-4256-9e7b-ea981d5c9926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:07.853215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1350/2689] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:22:07 2022] DELETE /identity/v3/users/2a06951cf07d47a1b4419daacdbd5078 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:22:07.857837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30b89c-1cbc-4975-8629-7ec165fad090 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:07.858134 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30b89c-1cbc-4975-8629-7ec165fad090 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:07.858433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e30b89c-1cbc-4975-8629-7ec165fad090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:07.875643 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f3d4c799-44ed-4256-9e7b-ea981d5c9926 None None] MFA Rules not processed for user `1c009a915aad4fe9b8c334e64ecb2e48`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:07.882617 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f3d4c799-44ed-4256-9e7b-ea981d5c9926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:07.896557 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-0e30b89c-1cbc-4975-8629-7ec165fad090 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:07.898667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0e30b89c-1cbc-4975-8629-7ec165fad090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:07.970940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1351/2690] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:22:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:22:07.974890 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1340/2691] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:22:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:22:07.979345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30b89c-1cbc-4975-8629-7ec165fad090 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:07.980897 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-0e30b89c-1cbc-4975-8629-7ec165fad090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:07.988760 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4c799-44ed-4256-9e7b-ea981d5c9926 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:07.991277 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f3d4c799-44ed-4256-9e7b-ea981d5c9926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:07.996180 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e30b89c-1cbc-4975-8629-7ec165fad090 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:08.000943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc92fb5-6569-4b70-aff8-f24c3543835e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:08.001626 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc92fb5-6569-4b70-aff8-f24c3543835e admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:08.001909 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc92fb5-6569-4b70-aff8-f24c3543835e admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:08.002180 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc92fb5-6569-4b70-aff8-f24c3543835e admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:08.002877 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc92fb5-6569-4b70-aff8-f24c3543835e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:08.004907 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc92fb5-6569-4b70-aff8-f24c3543835e admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:08.036558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1352/2692] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:22:07 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:22:08.037885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc92fb5-6569-4b70-aff8-f24c3543835e admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:08.053727 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:08.066428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d4c799-44ed-4256-9e7b-ea981d5c9926 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:08.090112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:08.111797 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc92fb5-6569-4b70-aff8-f24c3543835e admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:08.116871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22774a8b-fe99-40e5-bc31-ddb1a2c00a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:08.117583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22774a8b-fe99-40e5-bc31-ddb1a2c00a3a admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:08.117866 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22774a8b-fe99-40e5-bc31-ddb1a2c00a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:08.118141 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22774a8b-fe99-40e5-bc31-ddb1a2c00a3a admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:08.118834 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22774a8b-fe99-40e5-bc31-ddb1a2c00a3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:08.121920 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22774a8b-fe99-40e5-bc31-ddb1a2c00a3a admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:08.131852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5f2f4-135d-43f7-ad10-41870c22af49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:08.132448 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5f2f4-135d-43f7-ad10-41870c22af49 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:08.132946 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5f2f4-135d-43f7-ad10-41870c22af49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:08.133363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f5f2f4-135d-43f7-ad10-41870c22af49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:08.145590 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5f2f4-135d-43f7-ad10-41870c22af49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:08.147038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f5f2f4-135d-43f7-ad10-41870c22af49 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:08.153116 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1353/2693] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:22:08 2022] POST /identity/v3/users => generated 298 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:22:08.154613 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22774a8b-fe99-40e5-bc31-ddb1a2c00a3a admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:08.157676 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1341/2694] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:22:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:22:08.165700 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22774a8b-fe99-40e5-bc31-ddb1a2c00a3a admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:08.172167 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44997bb-5292-45f8-a4fe-a33937a78718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:08.173156 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44997bb-5292-45f8-a4fe-a33937a78718 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:08.173467 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44997bb-5292-45f8-a4fe-a33937a78718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:08.173742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44997bb-5292-45f8-a4fe-a33937a78718 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:08.174475 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44997bb-5292-45f8-a4fe-a33937a78718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:08.176209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44997bb-5292-45f8-a4fe-a33937a78718 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:08.188214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1354/2695] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:22:08.192375 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5f2f4-135d-43f7-ad10-41870c22af49 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:08.204214 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f5f2f4-135d-43f7-ad10-41870c22af49 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:08.209501 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeecd4e-db70-411b-bfc5-c6d873899472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:08.210686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeecd4e-db70-411b-bfc5-c6d873899472 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:08.211073 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeecd4e-db70-411b-bfc5-c6d873899472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:08.211673 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeecd4e-db70-411b-bfc5-c6d873899472 admin admin] PATH_INFO: `/v3/projects/93dc8dd0a6b54cfeaced481ced058522/users/57a7dd55fb42458998608966451cf3d1/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:08.233431 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeecd4e-db70-411b-bfc5-c6d873899472 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93dc8dd0a6b54cfeaced481ced058522, user_id=57a7dd55fb42458998608966451cf3d1, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:08.243062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeecd4e-db70-411b-bfc5-c6d873899472 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:08.274476 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1342/2696] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:22:08 2022] PUT /identity/v3/projects/93dc8dd0a6b54cfeaced481ced058522/users/57a7dd55fb42458998608966451cf3d1/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:22:08.276248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44997bb-5292-45f8-a4fe-a33937a78718 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:08.287547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44997bb-5292-45f8-a4fe-a33937a78718 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:08.371633 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addad344-f740-4867-8fcd-4bb69d64681c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:08.372332 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addad344-f740-4867-8fcd-4bb69d64681c admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:08.385373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addad344-f740-4867-8fcd-4bb69d64681c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:08.386008 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addad344-f740-4867-8fcd-4bb69d64681c admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:08.386933 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addad344-f740-4867-8fcd-4bb69d64681c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:08.388825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addad344-f740-4867-8fcd-4bb69d64681c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:08.404002 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1355/2697] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:22:08.407881 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeecd4e-db70-411b-bfc5-c6d873899472 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:08.437067 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeecd4e-db70-411b-bfc5-c6d873899472 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:08.451857 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d142cd-a7df-419b-9649-b631f845d4d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:08.453166 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d142cd-a7df-419b-9649-b631f845d4d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:08.453668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d142cd-a7df-419b-9649-b631f845d4d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:08.456774 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d142cd-a7df-419b-9649-b631f845d4d5 admin admin] PATH_INFO: `/v3/projects/93dc8dd0a6b54cfeaced481ced058522/users/57a7dd55fb42458998608966451cf3d1/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:08.459323 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d142cd-a7df-419b-9649-b631f845d4d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93dc8dd0a6b54cfeaced481ced058522, user_id=57a7dd55fb42458998608966451cf3d1, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:08.465549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d142cd-a7df-419b-9649-b631f845d4d5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:08.483226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1343/2698] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:22:08 2022] PUT /identity/v3/projects/93dc8dd0a6b54cfeaced481ced058522/users/57a7dd55fb42458998608966451cf3d1/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:22:08.567579 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addad344-f740-4867-8fcd-4bb69d64681c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:08.569747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-addad344-f740-4867-8fcd-4bb69d64681c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:08.585351 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addad344-f740-4867-8fcd-4bb69d64681c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:08.676427 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22627b53-891e-4dc6-9274-34e6d930fef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:08.677409 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22627b53-891e-4dc6-9274-34e6d930fef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:08.677682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22627b53-891e-4dc6-9274-34e6d930fef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:08.677955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22627b53-891e-4dc6-9274-34e6d930fef9 admin admin] PATH_INFO: `/v3/projects/0cd3780fb7a24fc3bc4c018f7dbc1f39` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:08.679354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22627b53-891e-4dc6-9274-34e6d930fef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cd3780fb7a24fc3bc4c018f7dbc1f39)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:08.681512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22627b53-891e-4dc6-9274-34e6d930fef9 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:08.720976 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:08.745971 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22627b53-891e-4dc6-9274-34e6d930fef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cd3780fb7a24fc3bc4c018f7dbc1f39', 'request_id': 'req-22627b53-891e-4dc6-9274-34e6d930fef9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:08.754030 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22627b53-891e-4dc6-9274-34e6d930fef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cd3780fb7a24fc3bc4c018f7dbc1f39', 'request_id': 'req-22627b53-891e-4dc6-9274-34e6d930fef9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:08.762519 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22627b53-891e-4dc6-9274-34e6d930fef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cd3780fb7a24fc3bc4c018f7dbc1f39', 'request_id': 'req-22627b53-891e-4dc6-9274-34e6d930fef9'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:08.779783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22627b53-891e-4dc6-9274-34e6d930fef9 admin admin] The token cache is being invalidate because project 0cd3780fb7a24fc3bc4c018f7dbc1f39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:22:08.780070 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22627b53-891e-4dc6-9274-34e6d930fef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:08.780994 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22627b53-891e-4dc6-9274-34e6d930fef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:08.785619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1356/2699] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:22:08 2022] DELETE /identity/v3/projects/0cd3780fb7a24fc3bc4c018f7dbc1f39 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:22:08.786814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d142cd-a7df-419b-9649-b631f845d4d5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:08.788845 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-76d142cd-a7df-419b-9649-b631f845d4d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:08.804580 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d142cd-a7df-419b-9649-b631f845d4d5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:08.880699 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86048932-542d-41c6-9adb-73fe884b4a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:08.881975 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86048932-542d-41c6-9adb-73fe884b4a85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:08.882385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86048932-542d-41c6-9adb-73fe884b4a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:08.883036 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86048932-542d-41c6-9adb-73fe884b4a85 admin admin] PATH_INFO: `/v3/users/57e40420ba5349fba285d586ed1d8e17` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:08.884580 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86048932-542d-41c6-9adb-73fe884b4a85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57e40420ba5349fba285d586ed1d8e17)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:08.887118 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86048932-542d-41c6-9adb-73fe884b4a85 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:08.960489 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-86048932-542d-41c6-9adb-73fe884b4a85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57e40420ba5349fba285d586ed1d8e17'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:08.961471 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-86048932-542d-41c6-9adb-73fe884b4a85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57e40420ba5349fba285d586ed1d8e17'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:08.976262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-86048932-542d-41c6-9adb-73fe884b4a85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57e40420ba5349fba285d586ed1d8e17'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:08.987051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-86048932-542d-41c6-9adb-73fe884b4a85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57e40420ba5349fba285d586ed1d8e17'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:08.998680 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-86048932-542d-41c6-9adb-73fe884b4a85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57e40420ba5349fba285d586ed1d8e17'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:09.010649 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1344/2700] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:22:08 2022] DELETE /identity/v3/users/57e40420ba5349fba285d586ed1d8e17 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:22:09.322219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22627b53-891e-4dc6-9274-34e6d930fef9 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:09.337283 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22627b53-891e-4dc6-9274-34e6d930fef9 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:09.422450 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f4297d-3168-4700-bf12-6230b1d3e70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:09.423648 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4297d-3168-4700-bf12-6230b1d3e70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:09.423994 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4297d-3168-4700-bf12-6230b1d3e70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:09.424404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f4297d-3168-4700-bf12-6230b1d3e70b admin admin] PATH_INFO: `/v3/projects/33aa7ee47aa84200b21799465e18f676` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:09.425669 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4297d-3168-4700-bf12-6230b1d3e70b admin admin] RBAC: Authorizing `identity:delete_project(project_id=33aa7ee47aa84200b21799465e18f676)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:09.427879 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f4297d-3168-4700-bf12-6230b1d3e70b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:09.486783 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-93f4297d-3168-4700-bf12-6230b1d3e70b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33aa7ee47aa84200b21799465e18f676', 'request_id': 'req-93f4297d-3168-4700-bf12-6230b1d3e70b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:09.494572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-93f4297d-3168-4700-bf12-6230b1d3e70b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33aa7ee47aa84200b21799465e18f676', 'request_id': 'req-93f4297d-3168-4700-bf12-6230b1d3e70b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:09.500979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-93f4297d-3168-4700-bf12-6230b1d3e70b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33aa7ee47aa84200b21799465e18f676', 'request_id': 'req-93f4297d-3168-4700-bf12-6230b1d3e70b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:09.518414 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-93f4297d-3168-4700-bf12-6230b1d3e70b admin admin] The token cache is being invalidate because project 33aa7ee47aa84200b21799465e18f676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:22:09.518414 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-93f4297d-3168-4700-bf12-6230b1d3e70b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:09.519051 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-93f4297d-3168-4700-bf12-6230b1d3e70b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:09.524275 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1357/2701] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:22:09 2022] DELETE /identity/v3/projects/33aa7ee47aa84200b21799465e18f676 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:22:09.526424 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59199a-1019-4634-b5bd-d96ae0bd89fb None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:09.527282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59199a-1019-4634-b5bd-d96ae0bd89fb None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:09.527784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f59199a-1019-4634-b5bd-d96ae0bd89fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:09.551339 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:09.563227 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-4f59199a-1019-4634-b5bd-d96ae0bd89fb None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:09.566174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-4f59199a-1019-4634-b5bd-d96ae0bd89fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:09.645895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1345/2702] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:22:09.647978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4c7ea-f6a4-47be-b76d-50993cbcb5a1 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:09.648269 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4c7ea-f6a4-47be-b76d-50993cbcb5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:09.648598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4c7ea-f6a4-47be-b76d-50993cbcb5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:09.668359 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-d7d4c7ea-f6a4-47be-b76d-50993cbcb5a1 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:09.669637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-d7d4c7ea-f6a4-47be-b76d-50993cbcb5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:09.677913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1358/2703] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:22:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:22:09.683245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59199a-1019-4634-b5bd-d96ae0bd89fb None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:09.694559 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f59199a-1019-4634-b5bd-d96ae0bd89fb None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:09.700568 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6e367-df4c-4637-ad9a-233488fbf23c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:09.701328 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6e367-df4c-4637-ad9a-233488fbf23c admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:09.701608 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6e367-df4c-4637-ad9a-233488fbf23c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:09.701886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c6e367-df4c-4637-ad9a-233488fbf23c admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:09.702928 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6e367-df4c-4637-ad9a-233488fbf23c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:22:09.702928 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6e367-df4c-4637-ad9a-233488fbf23c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:09.704552 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c6e367-df4c-4637-ad9a-233488fbf23c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:09.719945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1346/2704] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:22:09.724340 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4c7ea-f6a4-47be-b76d-50993cbcb5a1 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:09.735838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4c7ea-f6a4-47be-b76d-50993cbcb5a1 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:09.741556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3b546-92d3-4262-961b-a84f73a9c86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:09.742238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3b546-92d3-4262-961b-a84f73a9c86b admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:09.742526 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3b546-92d3-4262-961b-a84f73a9c86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:09.742803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3b546-92d3-4262-961b-a84f73a9c86b admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:09.744407 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3b546-92d3-4262-961b-a84f73a9c86b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:09.746949 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3b546-92d3-4262-961b-a84f73a9c86b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:09.786587 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6e367-df4c-4637-ad9a-233488fbf23c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:09.787041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1359/2705] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:22:09 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:22:09.799979 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c6e367-df4c-4637-ad9a-233488fbf23c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:09.877613 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9afcec-9e51-41f3-b634-ec6a922856a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:09.878549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9afcec-9e51-41f3-b634-ec6a922856a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:09.879328 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9afcec-9e51-41f3-b634-ec6a922856a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:09.879613 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9afcec-9e51-41f3-b634-ec6a922856a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:09.880770 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9afcec-9e51-41f3-b634-ec6a922856a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:09.882485 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9afcec-9e51-41f3-b634-ec6a922856a0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:09.912685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1347/2706] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:22:09 2022] POST /identity/v3/users => generated 286 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:22:09.917946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3b546-92d3-4262-961b-a84f73a9c86b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:09.932101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3b546-92d3-4262-961b-a84f73a9c86b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:09.937905 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520cfd2-c668-4a8c-b76c-c484af781e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:09.940848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520cfd2-c668-4a8c-b76c-c484af781e74 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:09.940848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520cfd2-c668-4a8c-b76c-c484af781e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:09.940848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e520cfd2-c668-4a8c-b76c-c484af781e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:09.940848 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520cfd2-c668-4a8c-b76c-c484af781e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:09.941904 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e520cfd2-c668-4a8c-b76c-c484af781e74 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:09.958538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9afcec-9e51-41f3-b634-ec6a922856a0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:09.965851 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1360/2707] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:22:09.972652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9afcec-9e51-41f3-b634-ec6a922856a0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:09.978110 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6007cf6-045b-4948-9efc-0819e00d6611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:09.978860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6007cf6-045b-4948-9efc-0819e00d6611 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:09.979148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6007cf6-045b-4948-9efc-0819e00d6611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:09.979660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6007cf6-045b-4948-9efc-0819e00d6611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:09.980460 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6007cf6-045b-4948-9efc-0819e00d6611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:09.982219 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6007cf6-045b-4948-9efc-0819e00d6611 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:10.003756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1348/2708] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:22:10.008723 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520cfd2-c668-4a8c-b76c-c484af781e74 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:10.025768 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e520cfd2-c668-4a8c-b76c-c484af781e74 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:10.033139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd0ef2-0d01-49cf-bedb-bb1618920b74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:10.034293 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd0ef2-0d01-49cf-bedb-bb1618920b74 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:10.034740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd0ef2-0d01-49cf-bedb-bb1618920b74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:10.034740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd0ef2-0d01-49cf-bedb-bb1618920b74 admin admin] PATH_INFO: `/v3/projects/8c70bd29ed724affb53fa8ce5a481429/users/4b8f062f488a4a74bf3232255d9ba2fd/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:10.037423 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:10.061926 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd0ef2-0d01-49cf-bedb-bb1618920b74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c70bd29ed724affb53fa8ce5a481429, user_id=4b8f062f488a4a74bf3232255d9ba2fd, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:10.064836 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd0ef2-0d01-49cf-bedb-bb1618920b74 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:10.085348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1361/2709] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:22:10 2022] PUT /identity/v3/projects/8c70bd29ed724affb53fa8ce5a481429/users/4b8f062f488a4a74bf3232255d9ba2fd/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:22:10.090951 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3c875-5ede-4f7b-ac73-e4d59a564063 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:10.092376 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3c875-5ede-4f7b-ac73-e4d59a564063 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:10.092733 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3c875-5ede-4f7b-ac73-e4d59a564063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:10.119413 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-79d3c875-5ede-4f7b-ac73-e4d59a564063 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:10.120892 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-79d3c875-5ede-4f7b-ac73-e4d59a564063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:10.211157 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1349/2710] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:22:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:22:10.223309 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd0ef2-0d01-49cf-bedb-bb1618920b74 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:10.225443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-dfdd0ef2-0d01-49cf-bedb-bb1618920b74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:10.357694 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd0ef2-0d01-49cf-bedb-bb1618920b74 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:10.375779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499dde2-2d30-4983-9f00-65bbed06213f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:10.376721 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499dde2-2d30-4983-9f00-65bbed06213f service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:10.377451 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499dde2-2d30-4983-9f00-65bbed06213f service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:10.377876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a499dde2-2d30-4983-9f00-65bbed06213f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:10.395504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499dde2-2d30-4983-9f00-65bbed06213f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:10.399394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a499dde2-2d30-4983-9f00-65bbed06213f service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:10.424512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1362/2711] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:22:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:22:11.572726 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4315e-1554-42bd-a2a4-cbb71dbfee24 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:11.573204 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4315e-1554-42bd-a2a4-cbb71dbfee24 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:11.573526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d4315e-1554-42bd-a2a4-cbb71dbfee24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:11.614211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:11.617987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-33d4315e-1554-42bd-a2a4-cbb71dbfee24 None None] MFA Rules not processed for user `4b8f062f488a4a74bf3232255d9ba2fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:11.620493 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-33d4315e-1554-42bd-a2a4-cbb71dbfee24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:11.723015 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1350/2712] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:22:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:22:11.727238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499dde2-2d30-4983-9f00-65bbed06213f service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:11.745088 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a499dde2-2d30-4983-9f00-65bbed06213f service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:11.753928 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce610ae6-0dc2-4060-b42c-804d2d4a9bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:11.755686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce610ae6-0dc2-4060-b42c-804d2d4a9bf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:11.756070 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce610ae6-0dc2-4060-b42c-804d2d4a9bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:11.756352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce610ae6-0dc2-4060-b42c-804d2d4a9bf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:11.757108 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce610ae6-0dc2-4060-b42c-804d2d4a9bf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:11.759188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce610ae6-0dc2-4060-b42c-804d2d4a9bf0 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:11.811191 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1363/2713] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:22:11 2022] POST /identity/v3/projects => generated 371 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:22:11.813338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4315e-1554-42bd-a2a4-cbb71dbfee24 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:11.827556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d4315e-1554-42bd-a2a4-cbb71dbfee24 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:11.925585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82a8bb-d6e4-4010-a319-d6241cbf31a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:11.926416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82a8bb-d6e4-4010-a319-d6241cbf31a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:11.927023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82a8bb-d6e4-4010-a319-d6241cbf31a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:11.927402 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82a8bb-d6e4-4010-a319-d6241cbf31a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:11.928205 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82a8bb-d6e4-4010-a319-d6241cbf31a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:11.931094 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82a8bb-d6e4-4010-a319-d6241cbf31a2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:11.959297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1351/2714] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:22:11 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:22:11.963422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce610ae6-0dc2-4060-b42c-804d2d4a9bf0 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:11.974404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce610ae6-0dc2-4060-b42c-804d2d4a9bf0 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:11.979926 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d98ec0-e492-4a39-a85a-08c71d74d887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:11.989564 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d98ec0-e492-4a39-a85a-08c71d74d887 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:11.990148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d98ec0-e492-4a39-a85a-08c71d74d887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:11.990632 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d98ec0-e492-4a39-a85a-08c71d74d887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:11.992034 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d98ec0-e492-4a39-a85a-08c71d74d887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:11.994803 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d98ec0-e492-4a39-a85a-08c71d74d887 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:12.013773 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1364/2715] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:22:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:22:12.015213 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82a8bb-d6e4-4010-a319-d6241cbf31a2 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:12.038766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82a8bb-d6e4-4010-a319-d6241cbf31a2 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:12.048843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c81a7-4b83-44eb-b71a-3f401a3fb41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:12.050254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c81a7-4b83-44eb-b71a-3f401a3fb41a admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:12.050706 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c81a7-4b83-44eb-b71a-3f401a3fb41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:12.051133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c81a7-4b83-44eb-b71a-3f401a3fb41a admin admin] PATH_INFO: `/v3/projects/e2091cd4e7d94578b1bf5eb48b174332/users/da454620f72a45d69a0ce32545a7c34b/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:12.070804 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c81a7-4b83-44eb-b71a-3f401a3fb41a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2091cd4e7d94578b1bf5eb48b174332, user_id=da454620f72a45d69a0ce32545a7c34b, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:12.074397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c81a7-4b83-44eb-b71a-3f401a3fb41a admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:12.110149 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1352/2716] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:22:12 2022] PUT /identity/v3/projects/e2091cd4e7d94578b1bf5eb48b174332/users/da454620f72a45d69a0ce32545a7c34b/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:22:12.111602 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d98ec0-e492-4a39-a85a-08c71d74d887 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:12.126808 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d98ec0-e492-4a39-a85a-08c71d74d887 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:12.241305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2064a57-7d2c-49b6-9ef9-72e85e0f1d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:12.242111 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2064a57-7d2c-49b6-9ef9-72e85e0f1d7d admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:12.242487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2064a57-7d2c-49b6-9ef9-72e85e0f1d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:12.242989 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2064a57-7d2c-49b6-9ef9-72e85e0f1d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:12.255077 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2064a57-7d2c-49b6-9ef9-72e85e0f1d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:12.257118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2064a57-7d2c-49b6-9ef9-72e85e0f1d7d admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:12.288074 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1365/2717] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:22:12.289749 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c81a7-4b83-44eb-b71a-3f401a3fb41a admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:12.306027 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c81a7-4b83-44eb-b71a-3f401a3fb41a admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:12.321618 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1e1d13-1ee4-4dcc-bd78-3d51a87a9954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:12.322843 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e1d13-1ee4-4dcc-bd78-3d51a87a9954 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:12.323216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e1d13-1ee4-4dcc-bd78-3d51a87a9954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:12.323931 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1e1d13-1ee4-4dcc-bd78-3d51a87a9954 admin admin] PATH_INFO: `/v3/projects/e2091cd4e7d94578b1bf5eb48b174332/users/da454620f72a45d69a0ce32545a7c34b/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:12.327598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e1d13-1ee4-4dcc-bd78-3d51a87a9954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2091cd4e7d94578b1bf5eb48b174332, user_id=da454620f72a45d69a0ce32545a7c34b, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:12.334282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1e1d13-1ee4-4dcc-bd78-3d51a87a9954 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:12.355177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1353/2718] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:22:12 2022] PUT /identity/v3/projects/e2091cd4e7d94578b1bf5eb48b174332/users/da454620f72a45d69a0ce32545a7c34b/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:22:13.546782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360b895-d07b-49ca-82c7-c83cd96bdb13 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:13.547372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360b895-d07b-49ca-82c7-c83cd96bdb13 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:13.548060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360b895-d07b-49ca-82c7-c83cd96bdb13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:13.549577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398a8d8-e90b-49fd-a92b-b3b51013a45b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:13.550000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398a8d8-e90b-49fd-a92b-b3b51013a45b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:13.550237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398a8d8-e90b-49fd-a92b-b3b51013a45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:13.580240 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7398a8d8-e90b-49fd-a92b-b3b51013a45b None None] MFA Rules not processed for user `da454620f72a45d69a0ce32545a7c34b`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:13.581419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-4360b895-d07b-49ca-82c7-c83cd96bdb13 None None] MFA Rules not processed for user `57a7dd55fb42458998608966451cf3d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:13.581632 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7398a8d8-e90b-49fd-a92b-b3b51013a45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:13.582733 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4360b895-d07b-49ca-82c7-c83cd96bdb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:13.639673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:13.643598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:13.677013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1366/2719] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:22:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:22:13.684121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1354/2720] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:22:13 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:22:13.691905 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360b895-d07b-49ca-82c7-c83cd96bdb13 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:13.693592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4360b895-d07b-49ca-82c7-c83cd96bdb13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:13.695222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398a8d8-e90b-49fd-a92b-b3b51013a45b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:13.697166 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7398a8d8-e90b-49fd-a92b-b3b51013a45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:13.768528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360b895-d07b-49ca-82c7-c83cd96bdb13 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:13.774044 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a5a52-2dda-4857-a039-2c621329e7a7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b5660cfd344dcd8abe14c88df686e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:13.774767 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a5a52-2dda-4857-a039-2c621329e7a7 service glance] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:13.775093 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a5a52-2dda-4857-a039-2c621329e7a7 service glance] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:13.775367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a5a52-2dda-4857-a039-2c621329e7a7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:13.777248 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-417a5a52-2dda-4857-a039-2c621329e7a7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:13.788484 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398a8d8-e90b-49fd-a92b-b3b51013a45b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:13.793384 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eef824-a766-48ea-9fb3-082df4029ad4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:13.794171 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eef824-a766-48ea-9fb3-082df4029ad4 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:13.794455 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eef824-a766-48ea-9fb3-082df4029ad4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:13.794730 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61eef824-a766-48ea-9fb3-082df4029ad4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:13.851932 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a5a52-2dda-4857-a039-2c621329e7a7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:13.853976 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a5a52-2dda-4857-a039-2c621329e7a7 service glance] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:13.864346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eef824-a766-48ea-9fb3-082df4029ad4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:13.866177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61eef824-a766-48ea-9fb3-082df4029ad4 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:13.867246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1367/2721] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:22:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:22:13.878653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1355/2722] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:22:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:22:14.043315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66633d-a50a-47d2-ae05-1d3269e81962 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:14.043955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66633d-a50a-47d2-ae05-1d3269e81962 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:14.044477 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de66633d-a50a-47d2-ae05-1d3269e81962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:14.066675 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-de66633d-a50a-47d2-ae05-1d3269e81962 None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:14.068774 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-de66633d-a50a-47d2-ae05-1d3269e81962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:14.148562 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1368/2723] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Jul 14 10:22:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:22:14.159269 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eef824-a766-48ea-9fb3-082df4029ad4 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:14.161019 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-61eef824-a766-48ea-9fb3-082df4029ad4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:14.300640 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61eef824-a766-48ea-9fb3-082df4029ad4 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:14.309611 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91a8e7-50f4-4c99-a8d6-0782464a8403 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:14.310546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91a8e7-50f4-4c99-a8d6-0782464a8403 service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:14.311022 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91a8e7-50f4-4c99-a8d6-0782464a8403 service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:14.311526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91a8e7-50f4-4c99-a8d6-0782464a8403 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:14.322728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91a8e7-50f4-4c99-a8d6-0782464a8403 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:14.324570 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb91a8e7-50f4-4c99-a8d6-0782464a8403 service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:14.340895 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1356/2724] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:22:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:22:15.365482 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66633d-a50a-47d2-ae05-1d3269e81962 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:15.367016 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-de66633d-a50a-47d2-ae05-1d3269e81962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:15.380965 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de66633d-a50a-47d2-ae05-1d3269e81962 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:15.391369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a39634-ddde-4505-87b7-9a6167aa0290 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:15.392049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a39634-ddde-4505-87b7-9a6167aa0290 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:15.392335 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a39634-ddde-4505-87b7-9a6167aa0290 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:15.392613 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a39634-ddde-4505-87b7-9a6167aa0290 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:15.406524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a39634-ddde-4505-87b7-9a6167aa0290 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:15.407978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a39634-ddde-4505-87b7-9a6167aa0290 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:15.422040 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1369/2725] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:22:15 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:22:23.536578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f751a-fb91-4176-8b64-7c7c6f6c1b3d None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:23.537063 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f751a-fb91-4176-8b64-7c7c6f6c1b3d None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:23.537462 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f751a-fb91-4176-8b64-7c7c6f6c1b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:23.546774 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=100 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:23.558651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-da2f751a-fb91-4176-8b64-7c7c6f6c1b3d None None] MFA Rules not processed for user `1c009a915aad4fe9b8c334e64ecb2e48`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:23.560146 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-da2f751a-fb91-4176-8b64-7c7c6f6c1b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:23.568739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1357/2726] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:22:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:22:25.363440 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d228d-9675-4dd5-b4f1-67e61b89abf7 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:25.363975 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d228d-9675-4dd5-b4f1-67e61b89abf7 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:25.363975 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d228d-9675-4dd5-b4f1-67e61b89abf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:25.383568 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6a5d228d-9675-4dd5-b4f1-67e61b89abf7 None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:25.385597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6a5d228d-9675-4dd5-b4f1-67e61b89abf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:25.393399 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1370/2727] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Jul 14 10:22:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:22:25.405334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f751a-fb91-4176-8b64-7c7c6f6c1b3d None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:25.417886 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f751a-fb91-4176-8b64-7c7c6f6c1b3d None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:25.422992 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091cf4f2-5b6c-4a9b-8725-3d1b087ba178 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:25.423777 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cf4f2-5b6c-4a9b-8725-3d1b087ba178 service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:25.424123 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cf4f2-5b6c-4a9b-8725-3d1b087ba178 service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:25.424464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091cf4f2-5b6c-4a9b-8725-3d1b087ba178 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:25.434242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091cf4f2-5b6c-4a9b-8725-3d1b087ba178 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:25.435860 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091cf4f2-5b6c-4a9b-8725-3d1b087ba178 service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:25.444345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1358/2728] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:22:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:22:29.189672 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d228d-9675-4dd5-b4f1-67e61b89abf7 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:29.208658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:29.307925 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d228d-9675-4dd5-b4f1-67e61b89abf7 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:29.312698 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645ecae-f0db-4e86-be3d-e73112979a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:29.313614 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645ecae-f0db-4e86-be3d-e73112979a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:29.313884 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645ecae-f0db-4e86-be3d-e73112979a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:29.314158 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645ecae-f0db-4e86-be3d-e73112979a19 admin admin] PATH_INFO: `/v3/users/4b8f062f488a4a74bf3232255d9ba2fd` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:29.315375 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645ecae-f0db-4e86-be3d-e73112979a19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b8f062f488a4a74bf3232255d9ba2fd)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:29.317338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1645ecae-f0db-4e86-be3d-e73112979a19 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:29.371109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1645ecae-f0db-4e86-be3d-e73112979a19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b8f062f488a4a74bf3232255d9ba2fd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:29.371841 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1645ecae-f0db-4e86-be3d-e73112979a19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b8f062f488a4a74bf3232255d9ba2fd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:29.385208 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1645ecae-f0db-4e86-be3d-e73112979a19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b8f062f488a4a74bf3232255d9ba2fd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:29.393155 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1645ecae-f0db-4e86-be3d-e73112979a19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b8f062f488a4a74bf3232255d9ba2fd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:29.402149 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-1645ecae-f0db-4e86-be3d-e73112979a19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b8f062f488a4a74bf3232255d9ba2fd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:29.408537 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1371/2729] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:22:29 2022] DELETE /identity/v3/users/4b8f062f488a4a74bf3232255d9ba2fd => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:22:29.595091 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091cf4f2-5b6c-4a9b-8725-3d1b087ba178 service swift] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:29.624516 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091cf4f2-5b6c-4a9b-8725-3d1b087ba178 service swift] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:29.686563 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9e9a5-ec37-40be-815b-d81b36cb5b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:29.687434 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9e9a5-ec37-40be-815b-d81b36cb5b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:29.687745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9e9a5-ec37-40be-815b-d81b36cb5b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:29.688017 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9e9a5-ec37-40be-815b-d81b36cb5b75 admin admin] PATH_INFO: `/v3/projects/8c70bd29ed724affb53fa8ce5a481429` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:29.689364 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9e9a5-ec37-40be-815b-d81b36cb5b75 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c70bd29ed724affb53fa8ce5a481429)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:29.691302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9e9a5-ec37-40be-815b-d81b36cb5b75 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:29.749247 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-87f9e9a5-ec37-40be-815b-d81b36cb5b75 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c70bd29ed724affb53fa8ce5a481429', 'request_id': 'req-87f9e9a5-ec37-40be-815b-d81b36cb5b75'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:29.755441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-87f9e9a5-ec37-40be-815b-d81b36cb5b75 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c70bd29ed724affb53fa8ce5a481429', 'request_id': 'req-87f9e9a5-ec37-40be-815b-d81b36cb5b75'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:29.764152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-87f9e9a5-ec37-40be-815b-d81b36cb5b75 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c70bd29ed724affb53fa8ce5a481429', 'request_id': 'req-87f9e9a5-ec37-40be-815b-d81b36cb5b75'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:29.786458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-87f9e9a5-ec37-40be-815b-d81b36cb5b75 admin admin] The token cache is being invalidate because project 8c70bd29ed724affb53fa8ce5a481429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:22:29.786792 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-87f9e9a5-ec37-40be-815b-d81b36cb5b75 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:29.787529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-87f9e9a5-ec37-40be-815b-d81b36cb5b75 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:29.789433 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1359/2730] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:22:29 2022] DELETE /identity/v3/projects/8c70bd29ed724affb53fa8ce5a481429 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:22:29.793430 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645ecae-f0db-4e86-be3d-e73112979a19 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:29.794836 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1645ecae-f0db-4e86-be3d-e73112979a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:29.810195 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645ecae-f0db-4e86-be3d-e73112979a19 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:29.869129 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962e75b-d50d-43e8-9c07-afaf34f0aab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:29.870351 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962e75b-d50d-43e8-9c07-afaf34f0aab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:29.870761 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962e75b-d50d-43e8-9c07-afaf34f0aab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:29.871036 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962e75b-d50d-43e8-9c07-afaf34f0aab6 admin admin] PATH_INFO: `/v3/users/da454620f72a45d69a0ce32545a7c34b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:29.872364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962e75b-d50d-43e8-9c07-afaf34f0aab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da454620f72a45d69a0ce32545a7c34b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:29.874626 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9962e75b-d50d-43e8-9c07-afaf34f0aab6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:29.918368 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:29.934795 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9962e75b-d50d-43e8-9c07-afaf34f0aab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da454620f72a45d69a0ce32545a7c34b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:29.935763 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9962e75b-d50d-43e8-9c07-afaf34f0aab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da454620f72a45d69a0ce32545a7c34b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:29.949417 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9962e75b-d50d-43e8-9c07-afaf34f0aab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da454620f72a45d69a0ce32545a7c34b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:29.956209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9962e75b-d50d-43e8-9c07-afaf34f0aab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da454620f72a45d69a0ce32545a7c34b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:29.964789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-9962e75b-d50d-43e8-9c07-afaf34f0aab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da454620f72a45d69a0ce32545a7c34b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:29.971742 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1372/2731] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:22:29 2022] DELETE /identity/v3/users/da454620f72a45d69a0ce32545a7c34b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:22:30.157577 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9e9a5-ec37-40be-815b-d81b36cb5b75 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:30.159278 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-87f9e9a5-ec37-40be-815b-d81b36cb5b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:30.176252 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9e9a5-ec37-40be-815b-d81b36cb5b75 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:30.250082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9a896-9346-452e-b6b8-70ec91bc3fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:30.251597 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9a896-9346-452e-b6b8-70ec91bc3fe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:30.253566 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9a896-9346-452e-b6b8-70ec91bc3fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:30.253853 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9a896-9346-452e-b6b8-70ec91bc3fe5 admin admin] PATH_INFO: `/v3/projects/e2091cd4e7d94578b1bf5eb48b174332` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:30.268334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9a896-9346-452e-b6b8-70ec91bc3fe5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2091cd4e7d94578b1bf5eb48b174332)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:30.268334 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9a896-9346-452e-b6b8-70ec91bc3fe5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:30.288586 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:30.344245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fac9a896-9346-452e-b6b8-70ec91bc3fe5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2091cd4e7d94578b1bf5eb48b174332', 'request_id': 'req-fac9a896-9346-452e-b6b8-70ec91bc3fe5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:30.357636 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fac9a896-9346-452e-b6b8-70ec91bc3fe5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2091cd4e7d94578b1bf5eb48b174332', 'request_id': 'req-fac9a896-9346-452e-b6b8-70ec91bc3fe5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:30.369237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fac9a896-9346-452e-b6b8-70ec91bc3fe5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2091cd4e7d94578b1bf5eb48b174332', 'request_id': 'req-fac9a896-9346-452e-b6b8-70ec91bc3fe5'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:30.383332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fac9a896-9346-452e-b6b8-70ec91bc3fe5 admin admin] The token cache is being invalidate because project e2091cd4e7d94578b1bf5eb48b174332 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:22:30.383623 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fac9a896-9346-452e-b6b8-70ec91bc3fe5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:30.384416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-fac9a896-9346-452e-b6b8-70ec91bc3fe5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:22:30.386294 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1360/2732] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:22:30 2022] DELETE /identity/v3/projects/e2091cd4e7d94578b1bf5eb48b174332 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:22:39.967417 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=16,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:40.379821 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=17,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:22:45.806588 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293c7fa-16ef-461d-b5bc-ebc4ce47ee45 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:45.807074 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293c7fa-16ef-461d-b5bc-ebc4ce47ee45 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:45.807264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293c7fa-16ef-461d-b5bc-ebc4ce47ee45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:45.825111 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-1293c7fa-16ef-461d-b5bc-ebc4ce47ee45 None None] MFA Rules not processed for user `7b0f433eba57485cbbfeab651a8e8b02`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:45.826372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1293c7fa-16ef-461d-b5bc-ebc4ce47ee45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:45.880159 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1373/2733] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:22:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:22:46.198093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9a896-9346-452e-b6b8-70ec91bc3fe5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:46.199498 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fac9a896-9346-452e-b6b8-70ec91bc3fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:46.267072 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9a896-9346-452e-b6b8-70ec91bc3fe5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:46.270951 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157c97c9-2b5a-4f82-9c25-d00a72f208e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:46.271602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c97c9-2b5a-4f82-9c25-d00a72f208e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:46.271882 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c97c9-2b5a-4f82-9c25-d00a72f208e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:46.272156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c97c9-2b5a-4f82-9c25-d00a72f208e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:46.274622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-157c97c9-2b5a-4f82-9c25-d00a72f208e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:46.335113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c97c9-2b5a-4f82-9c25-d00a72f208e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:46.336542 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157c97c9-2b5a-4f82-9c25-d00a72f208e6 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:46.345854 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1361/2734] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:22:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 14 10:22:53.010596 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293c7fa-16ef-461d-b5bc-ebc4ce47ee45 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:53.012019 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-1293c7fa-16ef-461d-b5bc-ebc4ce47ee45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:53.067255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293c7fa-16ef-461d-b5bc-ebc4ce47ee45 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:53.070837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7704051-744a-4823-86f9-cdade3cd3b68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:53.071474 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7704051-744a-4823-86f9-cdade3cd3b68 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:53.071756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7704051-744a-4823-86f9-cdade3cd3b68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:53.072030 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7704051-744a-4823-86f9-cdade3cd3b68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:53.073774 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-f7704051-744a-4823-86f9-cdade3cd3b68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:53.135584 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7704051-744a-4823-86f9-cdade3cd3b68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:53.137023 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7704051-744a-4823-86f9-cdade3cd3b68 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:53.146737 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1374/2735] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:22:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:22:53.311439 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7a0b5-2367-4265-9c4b-d3afdb617afa None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:53.311939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7a0b5-2367-4265-9c4b-d3afdb617afa None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:53.312043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7a0b5-2367-4265-9c4b-d3afdb617afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:53.329120 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-8fd7a0b5-2367-4265-9c4b-d3afdb617afa None None] MFA Rules not processed for user `8cd8f0490c6949cbba431b0b317683fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:22:53.330352 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8fd7a0b5-2367-4265-9c4b-d3afdb617afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:53.337942 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1362/2736] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:22:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:22:53.935043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7704051-744a-4823-86f9-cdade3cd3b68 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:22:53.943339 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7704051-744a-4823-86f9-cdade3cd3b68 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:22:53.947886 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b674394-6849-48fc-bfb2-35572ac00133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:22:53.948515 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b674394-6849-48fc-bfb2-35572ac00133 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:22:53.948791 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b674394-6849-48fc-bfb2-35572ac00133 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:22:53.949091 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b674394-6849-48fc-bfb2-35572ac00133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:22:53.950859 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5b674394-6849-48fc-bfb2-35572ac00133 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:22:53.960503 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b674394-6849-48fc-bfb2-35572ac00133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:22:53.961984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b674394-6849-48fc-bfb2-35572ac00133 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:22:53.969955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1375/2737] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:22:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:23:03.323858 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=51 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:23:03.966339 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=81 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:23:11.638810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7a0b5-2367-4265-9c4b-d3afdb617afa None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:23:11.649075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7a0b5-2367-4265-9c4b-d3afdb617afa None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:23:11.652511 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5946f40-743e-4d2a-b7bb-0d5ff4db1bf4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:23:11.653167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5946f40-743e-4d2a-b7bb-0d5ff4db1bf4 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:23:11.653440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5946f40-743e-4d2a-b7bb-0d5ff4db1bf4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:23:11.653704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5946f40-743e-4d2a-b7bb-0d5ff4db1bf4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:23:11.655438 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-b5946f40-743e-4d2a-b7bb-0d5ff4db1bf4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:23:11.716327 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5946f40-743e-4d2a-b7bb-0d5ff4db1bf4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:23:11.717799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5946f40-743e-4d2a-b7bb-0d5ff4db1bf4 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:23:11.728116 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1363/2738] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:23:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:23:21.723223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=30 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:23:47.012867 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b674394-6849-48fc-bfb2-35572ac00133 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:23:47.021958 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b674394-6849-48fc-bfb2-35572ac00133 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:23:47.025650 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29558ade-79c9-4851-a39e-c74b19699d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:23:47.026281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29558ade-79c9-4851-a39e-c74b19699d29 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:23:47.026559 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29558ade-79c9-4851-a39e-c74b19699d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:23:47.026828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29558ade-79c9-4851-a39e-c74b19699d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:23:47.028578 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-29558ade-79c9-4851-a39e-c74b19699d29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:23:47.115082 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29558ade-79c9-4851-a39e-c74b19699d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:23:47.116518 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29558ade-79c9-4851-a39e-c74b19699d29 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:23:47.139543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1376/2739] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:23:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:23:48.316810 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5946f40-743e-4d2a-b7bb-0d5ff4db1bf4 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:23:48.325888 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5946f40-743e-4d2a-b7bb-0d5ff4db1bf4 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:23:48.329638 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2c775-ba81-4b87-bd50-a1ff40f9db72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:23:48.330363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2c775-ba81-4b87-bd50-a1ff40f9db72 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:23:48.330651 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2c775-ba81-4b87-bd50-a1ff40f9db72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:23:48.330924 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc2c775-ba81-4b87-bd50-a1ff40f9db72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:23:48.332738 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8fc2c775-ba81-4b87-bd50-a1ff40f9db72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:23:48.346986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2c775-ba81-4b87-bd50-a1ff40f9db72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:23:48.348469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc2c775-ba81-4b87-bd50-a1ff40f9db72 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:23:48.359666 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1364/2740] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:23:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:23:57.134713 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=43 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:23:58.354750 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=6 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:24:04.610676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6a830-0be6-4503-990a-48b16ccfec8f None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:24:04.611136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6a830-0be6-4503-990a-48b16ccfec8f None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:24:04.611253 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6a830-0be6-4503-990a-48b16ccfec8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:24:04.661789 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-6ec6a830-0be6-4503-990a-48b16ccfec8f None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:24:04.663020 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-6ec6a830-0be6-4503-990a-48b16ccfec8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:24:04.720301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1377/2741] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Jul 14 10:24:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:24:04.763409 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2c775-ba81-4b87-bd50-a1ff40f9db72 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:24:04.763894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-8fc2c775-ba81-4b87-bd50-a1ff40f9db72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:24:04.842238 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc2c775-ba81-4b87-bd50-a1ff40f9db72 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:24:04.847325 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526f0af-b77b-4bc5-bbd6-09aa69db065b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:24:04.848516 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526f0af-b77b-4bc5-bbd6-09aa69db065b service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:24:04.848799 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526f0af-b77b-4bc5-bbd6-09aa69db065b service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:24:04.849155 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1526f0af-b77b-4bc5-bbd6-09aa69db065b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:24:04.858710 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526f0af-b77b-4bc5-bbd6-09aa69db065b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:24:04.860280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1526f0af-b77b-4bc5-bbd6-09aa69db065b service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:24:04.869686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1365/2742] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:24:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:24:14.714216 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=36 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:24:14.866503 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=32 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:24:20.043925 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db05128-f4d9-4512-a5d7-0e21504492a3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:24:20.044446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db05128-f4d9-4512-a5d7-0e21504492a3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:24:20.044446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db05128-f4d9-4512-a5d7-0e21504492a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:24:20.063890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-3db05128-f4d9-4512-a5d7-0e21504492a3 None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:24:20.065146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-3db05128-f4d9-4512-a5d7-0e21504492a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:24:20.114939 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1378/2743] 10.222.0.61 () {58 vars in 1009 bytes} [Thu Jul 14 10:24:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:24:20.127016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526f0af-b77b-4bc5-bbd6-09aa69db065b service swift] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:24:20.152122 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1526f0af-b77b-4bc5-bbd6-09aa69db065b service swift] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:24:20.159468 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012c1110-a313-4119-a509-6262fc32f557 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:24:20.160580 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c1110-a313-4119-a509-6262fc32f557 service swift] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:24:20.161201 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c1110-a313-4119-a509-6262fc32f557 service swift] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:24:20.161699 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012c1110-a313-4119-a509-6262fc32f557 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:24:20.195329 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c1110-a313-4119-a509-6262fc32f557 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:24:20.196947 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012c1110-a313-4119-a509-6262fc32f557 service swift] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:24:20.206940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1366/2744] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:24:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:24:30.105136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=7 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:24:30.202522 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=10 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:24:30.380369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db05128-f4d9-4512-a5d7-0e21504492a3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:24:30.389935 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db05128-f4d9-4512-a5d7-0e21504492a3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:24:30.394243 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63dab7-f24e-4690-981f-00502115b72c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:24:30.394913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63dab7-f24e-4690-981f-00502115b72c service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:24:30.395194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63dab7-f24e-4690-981f-00502115b72c service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:24:30.395467 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63dab7-f24e-4690-981f-00502115b72c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:24:30.397242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5f63dab7-f24e-4690-981f-00502115b72c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:24:30.407056 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63dab7-f24e-4690-981f-00502115b72c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:24:30.408529 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63dab7-f24e-4690-981f-00502115b72c service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:24:30.417314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1379/2745] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:24:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:24:35.118790 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00578693-e95f-45b2-a505-707422508aea None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:24:35.119315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00578693-e95f-45b2-a505-707422508aea None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:24:35.119315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00578693-e95f-45b2-a505-707422508aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:24:35.138616 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-00578693-e95f-45b2-a505-707422508aea None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:24:35.139838 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-00578693-e95f-45b2-a505-707422508aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:24:35.147642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1367/2746] 10.222.0.61 () {58 vars in 967 bytes} [Thu Jul 14 10:24:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:24:35.160238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63dab7-f24e-4690-981f-00502115b72c service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:24:35.172207 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63dab7-f24e-4690-981f-00502115b72c service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:24:35.177688 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab5757-758f-4255-872b-eacbd9cda825 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:24:35.178315 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab5757-758f-4255-872b-eacbd9cda825 service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:24:35.178592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab5757-758f-4255-872b-eacbd9cda825 service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:24:35.178858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ab5757-758f-4255-872b-eacbd9cda825 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:24:35.186655 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab5757-758f-4255-872b-eacbd9cda825 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:24:35.188247 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ab5757-758f-4255-872b-eacbd9cda825 service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:24:35.196656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1380/2747] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:24:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:24:35.892059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4461e0-b931-42a8-984b-4d3251f2f877 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:24:35.892400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4461e0-b931-42a8-984b-4d3251f2f877 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:24:35.893197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4461e0-b931-42a8-984b-4d3251f2f877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:24:35.911664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-fd4461e0-b931-42a8-984b-4d3251f2f877 None None] MFA Rules not processed for user `dd2b9ad7efa848938627385b411727ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:24:35.913053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fd4461e0-b931-42a8-984b-4d3251f2f877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:24:35.920127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1368/2748] 10.222.0.61 () {58 vars in 967 bytes} [Thu Jul 14 10:24:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:24:35.932532 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab5757-758f-4255-872b-eacbd9cda825 service swift] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:24:35.942922 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ab5757-758f-4255-872b-eacbd9cda825 service swift] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:24:35.947913 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b211e-3d7b-4529-a3e1-e80cf874feb9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5fa502e8b014c719fcece0a130fdd7e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:24:35.948542 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b211e-3d7b-4529-a3e1-e80cf874feb9 service swift] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:24:35.948823 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b211e-3d7b-4529-a3e1-e80cf874feb9 service swift] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:24:35.949139 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b211e-3d7b-4529-a3e1-e80cf874feb9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:24:35.958446 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b211e-3d7b-4529-a3e1-e80cf874feb9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:24:35.959588 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b211e-3d7b-4529-a3e1-e80cf874feb9 service swift] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:24:35.967689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1381/2749] 10.222.0.61 () {60 vars in 1384 bytes} [Thu Jul 14 10:24:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 14 10:24:42.191965 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4461e0-b931-42a8-984b-4d3251f2f877 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:24:42.193454 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-fd4461e0-b931-42a8-984b-4d3251f2f877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:24:42.257919 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4461e0-b931-42a8-984b-4d3251f2f877 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:24:42.262698 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199c696-0972-4b3d-9bba-25b22c097638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:24:42.263505 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199c696-0972-4b3d-9bba-25b22c097638 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:24:42.263779 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199c696-0972-4b3d-9bba-25b22c097638 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:24:42.264045 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199c696-0972-4b3d-9bba-25b22c097638 admin admin] PATH_INFO: `/v3/users/1c009a915aad4fe9b8c334e64ecb2e48` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:24:42.265266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199c696-0972-4b3d-9bba-25b22c097638 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c009a915aad4fe9b8c334e64ecb2e48)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:24:42.267203 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199c696-0972-4b3d-9bba-25b22c097638 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:24:42.316897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9199c696-0972-4b3d-9bba-25b22c097638 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c009a915aad4fe9b8c334e64ecb2e48'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:42.317585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9199c696-0972-4b3d-9bba-25b22c097638 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c009a915aad4fe9b8c334e64ecb2e48'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:42.332446 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9199c696-0972-4b3d-9bba-25b22c097638 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c009a915aad4fe9b8c334e64ecb2e48'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:42.339664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9199c696-0972-4b3d-9bba-25b22c097638 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c009a915aad4fe9b8c334e64ecb2e48'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:42.347832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-9199c696-0972-4b3d-9bba-25b22c097638 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c009a915aad4fe9b8c334e64ecb2e48'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:42.354079 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1369/2750] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:24:42 2022] DELETE /identity/v3/users/1c009a915aad4fe9b8c334e64ecb2e48 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:24:42.527616 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b211e-3d7b-4529-a3e1-e80cf874feb9 service swift] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:24:42.539433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b211e-3d7b-4529-a3e1-e80cf874feb9 service swift] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:24:42.595740 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:24:42.596637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:24:42.596943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:24:42.597229 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc admin admin] PATH_INFO: `/v3/projects/936db5c6e71f4012a4838d484c1ca92e` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:24:42.598693 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=936db5c6e71f4012a4838d484c1ca92e)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:24:42.600589 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:24:42.648031 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '936db5c6e71f4012a4838d484c1ca92e', 'request_id': 'req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:42.653941 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '936db5c6e71f4012a4838d484c1ca92e', 'request_id': 'req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:42.659651 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '936db5c6e71f4012a4838d484c1ca92e', 'request_id': 'req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:42.671873 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc admin admin] The token cache is being invalidate because project 936db5c6e71f4012a4838d484c1ca92e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:24:42.672161 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:42.672785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:42.674917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1382/2751] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:24:42 2022] DELETE /identity/v3/projects/936db5c6e71f4012a4838d484c1ca92e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:24:42.679033 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199c696-0972-4b3d-9bba-25b22c097638 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:24:42.680447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-9199c696-0972-4b3d-9bba-25b22c097638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:24:42.694280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199c696-0972-4b3d-9bba-25b22c097638 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:24:42.749471 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406da71e-5e8e-4da4-8f37-5b28bbf8350c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:24:42.750340 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406da71e-5e8e-4da4-8f37-5b28bbf8350c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:24:42.750619 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406da71e-5e8e-4da4-8f37-5b28bbf8350c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:24:42.750889 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406da71e-5e8e-4da4-8f37-5b28bbf8350c admin admin] PATH_INFO: `/v3/users/57a7dd55fb42458998608966451cf3d1` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:24:42.752635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406da71e-5e8e-4da4-8f37-5b28bbf8350c admin admin] RBAC: Authorizing `identity:delete_user(user_id=57a7dd55fb42458998608966451cf3d1)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:24:42.754621 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406da71e-5e8e-4da4-8f37-5b28bbf8350c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:24:42.760117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:24:42.799568 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-406da71e-5e8e-4da4-8f37-5b28bbf8350c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57a7dd55fb42458998608966451cf3d1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:42.800209 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-406da71e-5e8e-4da4-8f37-5b28bbf8350c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57a7dd55fb42458998608966451cf3d1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:42.811972 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-406da71e-5e8e-4da4-8f37-5b28bbf8350c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57a7dd55fb42458998608966451cf3d1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:42.819266 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-406da71e-5e8e-4da4-8f37-5b28bbf8350c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57a7dd55fb42458998608966451cf3d1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:42.825947 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-406da71e-5e8e-4da4-8f37-5b28bbf8350c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57a7dd55fb42458998608966451cf3d1'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:42.832157 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1370/2752] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:24:42 2022] DELETE /identity/v3/users/57a7dd55fb42458998608966451cf3d1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:24:43.001148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:24:43.003130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:24:43.016558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f3b6b-4ee1-4ae2-b028-bf3a9c5e74cc admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:24:43.068477 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:24:43.079876 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c11427-aecc-4772-b799-6122980d4d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:24:43.080686 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c11427-aecc-4772-b799-6122980d4d46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:24:43.080994 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c11427-aecc-4772-b799-6122980d4d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:24:43.081267 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c11427-aecc-4772-b799-6122980d4d46 admin admin] PATH_INFO: `/v3/projects/93dc8dd0a6b54cfeaced481ced058522` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:24:43.082443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c11427-aecc-4772-b799-6122980d4d46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93dc8dd0a6b54cfeaced481ced058522)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:24:43.084354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c11427-aecc-4772-b799-6122980d4d46 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:24:43.131705 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22c11427-aecc-4772-b799-6122980d4d46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93dc8dd0a6b54cfeaced481ced058522', 'request_id': 'req-22c11427-aecc-4772-b799-6122980d4d46'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:43.138032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22c11427-aecc-4772-b799-6122980d4d46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93dc8dd0a6b54cfeaced481ced058522', 'request_id': 'req-22c11427-aecc-4772-b799-6122980d4d46'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:43.143542 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22c11427-aecc-4772-b799-6122980d4d46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93dc8dd0a6b54cfeaced481ced058522', 'request_id': 'req-22c11427-aecc-4772-b799-6122980d4d46'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:43.155260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22c11427-aecc-4772-b799-6122980d4d46 admin admin] The token cache is being invalidate because project 93dc8dd0a6b54cfeaced481ced058522 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:24:43.155542 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22c11427-aecc-4772-b799-6122980d4d46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:43.156152 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22c11427-aecc-4772-b799-6122980d4d46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:24:43.158377 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1383/2753] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:24:43 2022] DELETE /identity/v3/projects/93dc8dd0a6b54cfeaced481ced058522 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:24:52.828614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:24:53.152304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:24:55.694716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406da71e-5e8e-4da4-8f37-5b28bbf8350c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:24:55.696243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-406da71e-5e8e-4da4-8f37-5b28bbf8350c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:24:55.774331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406da71e-5e8e-4da4-8f37-5b28bbf8350c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:24:55.836380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451ecfd-38a7-47b9-a27d-5498c8fe1759 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:24:55.837072 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451ecfd-38a7-47b9-a27d-5498c8fe1759 service nova] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:24:55.837360 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451ecfd-38a7-47b9-a27d-5498c8fe1759 service nova] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:24:55.837639 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5451ecfd-38a7-47b9-a27d-5498c8fe1759 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:24:55.839547 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5451ecfd-38a7-47b9-a27d-5498c8fe1759 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:24:55.852136 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451ecfd-38a7-47b9-a27d-5498c8fe1759 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:24:55.853757 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5451ecfd-38a7-47b9-a27d-5498c8fe1759 service nova] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:24:55.865044 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1371/2754] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:24:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:25:05.860455 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=62 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:05.983261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c11427-aecc-4772-b799-6122980d4d46 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:05.986057 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-22c11427-aecc-4772-b799-6122980d4d46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:06.051241 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c11427-aecc-4772-b799-6122980d4d46 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:06.054863 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f799b-2057-4e34-a1e3-d61f47fce866 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:06.055537 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f799b-2057-4e34-a1e3-d61f47fce866 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:06.055844 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f799b-2057-4e34-a1e3-d61f47fce866 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:06.056138 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f799b-2057-4e34-a1e3-d61f47fce866 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:06.058055 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-161f799b-2057-4e34-a1e3-d61f47fce866 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:06.123340 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f799b-2057-4e34-a1e3-d61f47fce866 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:06.124779 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f799b-2057-4e34-a1e3-d61f47fce866 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:06.134912 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1384/2755] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:25:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:25:09.260987 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451ecfd-38a7-47b9-a27d-5498c8fe1759 service nova] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:09.264049 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5451ecfd-38a7-47b9-a27d-5498c8fe1759 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:09.328836 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5451ecfd-38a7-47b9-a27d-5498c8fe1759 service nova] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:09.334852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134c71a-3e14-4302-8e33-318d17a3b44b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:09.335513 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134c71a-3e14-4302-8e33-318d17a3b44b service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:09.335924 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134c71a-3e14-4302-8e33-318d17a3b44b service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:09.336223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0134c71a-3e14-4302-8e33-318d17a3b44b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:09.338118 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0134c71a-3e14-4302-8e33-318d17a3b44b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:09.408341 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134c71a-3e14-4302-8e33-318d17a3b44b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:09.409771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0134c71a-3e14-4302-8e33-318d17a3b44b service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:09.419400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1372/2756] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:25:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:25:12.350205 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f799b-2057-4e34-a1e3-d61f47fce866 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:12.356101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-161f799b-2057-4e34-a1e3-d61f47fce866 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:12.457289 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134c71a-3e14-4302-8e33-318d17a3b44b service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:12.458760 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0134c71a-3e14-4302-8e33-318d17a3b44b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:12.481128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0134c71a-3e14-4302-8e33-318d17a3b44b service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:12.491493 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbb4f5-cbdd-44c8-a0af-d8bd6695af86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:12.492635 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbb4f5-cbdd-44c8-a0af-d8bd6695af86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:12.493827 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbb4f5-cbdd-44c8-a0af-d8bd6695af86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:12.494913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbb4f5-cbdd-44c8-a0af-d8bd6695af86 admin admin] PATH_INFO: `/v3/users/8cd8f0490c6949cbba431b0b317683fd` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:12.496449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbb4f5-cbdd-44c8-a0af-d8bd6695af86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cd8f0490c6949cbba431b0b317683fd)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:12.499705 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbb4f5-cbdd-44c8-a0af-d8bd6695af86 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:12.526331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f799b-2057-4e34-a1e3-d61f47fce866 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:12.536503 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06db7b6-64cc-45bb-a137-ba89d80acda3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff4c42a4f4f549378e5bc1cd92f801c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:12.537259 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06db7b6-64cc-45bb-a137-ba89d80acda3 service placement] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:12.537557 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06db7b6-64cc-45bb-a137-ba89d80acda3 service placement] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:12.537796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06db7b6-64cc-45bb-a137-ba89d80acda3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:12.541434 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e06db7b6-64cc-45bb-a137-ba89d80acda3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:12.557223 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06db7b6-64cc-45bb-a137-ba89d80acda3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:12.559122 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06db7b6-64cc-45bb-a137-ba89d80acda3 service placement] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:12.572612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1385/2757] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:25:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:25:12.573994 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06db7b6-64cc-45bb-a137-ba89d80acda3 service placement] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:12.575835 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-e06db7b6-64cc-45bb-a137-ba89d80acda3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:12.604786 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06db7b6-64cc-45bb-a137-ba89d80acda3 service placement] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:12.609619 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e6fbb4f5-cbdd-44c8-a0af-d8bd6695af86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cd8f0490c6949cbba431b0b317683fd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:12.611170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e6fbb4f5-cbdd-44c8-a0af-d8bd6695af86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cd8f0490c6949cbba431b0b317683fd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:12.613295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b6661-d297-4702-88a9-fcdf45f07824 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e9f4efa3e2f343c2ac858a277601a7df', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:12.614322 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b6661-d297-4702-88a9-fcdf45f07824 service nova] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:12.614922 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b6661-d297-4702-88a9-fcdf45f07824 service nova] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:12.615456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6b6661-d297-4702-88a9-fcdf45f07824 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:12.618843 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2b6b6661-d297-4702-88a9-fcdf45f07824 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:12.634730 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b6661-d297-4702-88a9-fcdf45f07824 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:12.636763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e6fbb4f5-cbdd-44c8-a0af-d8bd6695af86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cd8f0490c6949cbba431b0b317683fd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:12.636946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6b6661-d297-4702-88a9-fcdf45f07824 service nova] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:12.652707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e6fbb4f5-cbdd-44c8-a0af-d8bd6695af86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cd8f0490c6949cbba431b0b317683fd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:12.665217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1386/2758] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:25:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:25:12.669958 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e6fbb4f5-cbdd-44c8-a0af-d8bd6695af86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cd8f0490c6949cbba431b0b317683fd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:12.680604 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1373/2759] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:25:12 2022] DELETE /identity/v3/users/8cd8f0490c6949cbba431b0b317683fd => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:25:12.871600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b6661-d297-4702-88a9-fcdf45f07824 service nova] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:12.879961 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2b6b6661-d297-4702-88a9-fcdf45f07824 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:12.906953 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6b6661-d297-4702-88a9-fcdf45f07824 service nova] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:12.962443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=100 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:13.002449 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5134d9-e99a-4966-9742-ece41cc78bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:13.003477 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5134d9-e99a-4966-9742-ece41cc78bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:13.003627 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5134d9-e99a-4966-9742-ece41cc78bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:13.003843 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5134d9-e99a-4966-9742-ece41cc78bb2 admin admin] PATH_INFO: `/v3/projects/f5484157d3894ec097d1f1b2e0bacc16` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:13.005115 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5134d9-e99a-4966-9742-ece41cc78bb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5484157d3894ec097d1f1b2e0bacc16)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:13.006997 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5134d9-e99a-4966-9742-ece41cc78bb2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:13.064634 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8e5134d9-e99a-4966-9742-ece41cc78bb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5484157d3894ec097d1f1b2e0bacc16', 'request_id': 'req-8e5134d9-e99a-4966-9742-ece41cc78bb2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:13.072039 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8e5134d9-e99a-4966-9742-ece41cc78bb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5484157d3894ec097d1f1b2e0bacc16', 'request_id': 'req-8e5134d9-e99a-4966-9742-ece41cc78bb2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:13.079700 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8e5134d9-e99a-4966-9742-ece41cc78bb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5484157d3894ec097d1f1b2e0bacc16', 'request_id': 'req-8e5134d9-e99a-4966-9742-ece41cc78bb2'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:13.092793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8e5134d9-e99a-4966-9742-ece41cc78bb2 admin admin] The token cache is being invalidate because project f5484157d3894ec097d1f1b2e0bacc16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:25:13.093107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8e5134d9-e99a-4966-9742-ece41cc78bb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:13.094246 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8e5134d9-e99a-4966-9742-ece41cc78bb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:13.095984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1387/2760] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:25:12 2022] DELETE /identity/v3/projects/f5484157d3894ec097d1f1b2e0bacc16 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:13.099892 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbb4f5-cbdd-44c8-a0af-d8bd6695af86 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:13.110652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbb4f5-cbdd-44c8-a0af-d8bd6695af86 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:13.136329 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:13.173739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88108e2b-2162-4d42-82e7-c82dae304c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:13.174738 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88108e2b-2162-4d42-82e7-c82dae304c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:13.175126 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88108e2b-2162-4d42-82e7-c82dae304c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:13.175401 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88108e2b-2162-4d42-82e7-c82dae304c25 admin admin] PATH_INFO: `/v3/users/8463a3a31a4f42b18c26aefd8cbe4844` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:13.177241 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88108e2b-2162-4d42-82e7-c82dae304c25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8463a3a31a4f42b18c26aefd8cbe4844)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:13.179164 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88108e2b-2162-4d42-82e7-c82dae304c25 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:13.268612 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-88108e2b-2162-4d42-82e7-c82dae304c25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8463a3a31a4f42b18c26aefd8cbe4844'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:13.269457 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-88108e2b-2162-4d42-82e7-c82dae304c25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8463a3a31a4f42b18c26aefd8cbe4844'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:13.287396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-88108e2b-2162-4d42-82e7-c82dae304c25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8463a3a31a4f42b18c26aefd8cbe4844'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:13.296185 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-88108e2b-2162-4d42-82e7-c82dae304c25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8463a3a31a4f42b18c26aefd8cbe4844'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:13.307553 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-88108e2b-2162-4d42-82e7-c82dae304c25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8463a3a31a4f42b18c26aefd8cbe4844'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:13.315580 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1374/2761] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:25:13 2022] DELETE /identity/v3/users/8463a3a31a4f42b18c26aefd8cbe4844 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:13.535964 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5134d9-e99a-4966-9742-ece41cc78bb2 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:13.537546 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8e5134d9-e99a-4966-9742-ece41cc78bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:13.551251 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5134d9-e99a-4966-9742-ece41cc78bb2 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:13.648117 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:13.648916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:13.649262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:13.649557 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da admin admin] PATH_INFO: `/v3/projects/a7f90d9e3e7e463a97212fe1d8dca158` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:13.650774 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7f90d9e3e7e463a97212fe1d8dca158)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:13.653060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:13.709354 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7f90d9e3e7e463a97212fe1d8dca158', 'request_id': 'req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:13.715318 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7f90d9e3e7e463a97212fe1d8dca158', 'request_id': 'req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:13.720857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:13.723366 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7f90d9e3e7e463a97212fe1d8dca158', 'request_id': 'req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:13.741212 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da admin admin] The token cache is being invalidate because project a7f90d9e3e7e463a97212fe1d8dca158 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:25:13.741504 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:13.742133 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:13.743751 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1388/2762] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:25:13 2022] DELETE /identity/v3/projects/a7f90d9e3e7e463a97212fe1d8dca158 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:16.507259 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88108e2b-2162-4d42-82e7-c82dae304c25 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:16.508971 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-88108e2b-2162-4d42-82e7-c82dae304c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:16.566215 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88108e2b-2162-4d42-82e7-c82dae304c25 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:16.613546 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee05d98-aab9-43bb-8b46-49ef786a2def service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:16.614187 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee05d98-aab9-43bb-8b46-49ef786a2def service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:16.614465 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee05d98-aab9-43bb-8b46-49ef786a2def service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:16.614737 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee05d98-aab9-43bb-8b46-49ef786a2def service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:16.616548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0ee05d98-aab9-43bb-8b46-49ef786a2def service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:16.629568 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:16.680654 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee05d98-aab9-43bb-8b46-49ef786a2def service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:16.682129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee05d98-aab9-43bb-8b46-49ef786a2def service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:16.692861 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1375/2763] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:25:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:25:19.886978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:19.888622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:19.901467 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d5ba3-f2ae-456c-ac48-5510f8fc45da admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:19.906990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22df4058-c595-4122-a56d-398aee5dc4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:19.907871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df4058-c595-4122-a56d-398aee5dc4fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:19.908148 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df4058-c595-4122-a56d-398aee5dc4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:19.908419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22df4058-c595-4122-a56d-398aee5dc4fd admin admin] PATH_INFO: `/v3/users/7b0f433eba57485cbbfeab651a8e8b02` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:19.909693 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df4058-c595-4122-a56d-398aee5dc4fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b0f433eba57485cbbfeab651a8e8b02)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:19.911637 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22df4058-c595-4122-a56d-398aee5dc4fd admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:19.961396 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22df4058-c595-4122-a56d-398aee5dc4fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b0f433eba57485cbbfeab651a8e8b02'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:19.962062 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22df4058-c595-4122-a56d-398aee5dc4fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b0f433eba57485cbbfeab651a8e8b02'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:19.975376 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22df4058-c595-4122-a56d-398aee5dc4fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b0f433eba57485cbbfeab651a8e8b02'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:19.982454 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22df4058-c595-4122-a56d-398aee5dc4fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b0f433eba57485cbbfeab651a8e8b02'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:19.990729 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-22df4058-c595-4122-a56d-398aee5dc4fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b0f433eba57485cbbfeab651a8e8b02'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:19.999945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1389/2764] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:25:19 2022] DELETE /identity/v3/users/7b0f433eba57485cbbfeab651a8e8b02 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:20.179015 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee05d98-aab9-43bb-8b46-49ef786a2def service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:20.190264 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee05d98-aab9-43bb-8b46-49ef786a2def service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:20.260580 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6b4ab-78b5-440b-bd98-1bd0016fe7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:20.261565 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6b4ab-78b5-440b-bd98-1bd0016fe7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:20.261884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6b4ab-78b5-440b-bd98-1bd0016fe7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:20.262191 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6b4ab-78b5-440b-bd98-1bd0016fe7bd admin admin] PATH_INFO: `/v3/projects/4f945783031448849dc0e5a3337f8c76` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:20.263769 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6b4ab-78b5-440b-bd98-1bd0016fe7bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f945783031448849dc0e5a3337f8c76)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:20.268458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6b4ab-78b5-440b-bd98-1bd0016fe7bd admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:20.322642 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cbc6b4ab-78b5-440b-bd98-1bd0016fe7bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f945783031448849dc0e5a3337f8c76', 'request_id': 'req-cbc6b4ab-78b5-440b-bd98-1bd0016fe7bd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:20.329748 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cbc6b4ab-78b5-440b-bd98-1bd0016fe7bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f945783031448849dc0e5a3337f8c76', 'request_id': 'req-cbc6b4ab-78b5-440b-bd98-1bd0016fe7bd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:20.338075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cbc6b4ab-78b5-440b-bd98-1bd0016fe7bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f945783031448849dc0e5a3337f8c76', 'request_id': 'req-cbc6b4ab-78b5-440b-bd98-1bd0016fe7bd'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:20.350117 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cbc6b4ab-78b5-440b-bd98-1bd0016fe7bd admin admin] The token cache is being invalidate because project 4f945783031448849dc0e5a3337f8c76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:25:20.350403 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cbc6b4ab-78b5-440b-bd98-1bd0016fe7bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:20.351050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cbc6b4ab-78b5-440b-bd98-1bd0016fe7bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:20.352581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1376/2765] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:25:20 2022] DELETE /identity/v3/projects/4f945783031448849dc0e5a3337f8c76 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:20.359189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f408e99-9af5-4491-8e83-e64894723b82 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:20.359553 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f408e99-9af5-4491-8e83-e64894723b82 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:20.359896 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f408e99-9af5-4491-8e83-e64894723b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:20.378826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7f408e99-9af5-4491-8e83-e64894723b82 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:25:20.380188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7f408e99-9af5-4491-8e83-e64894723b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:20.452064 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1390/2766] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:25:20.460501 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13261d5a-4202-466e-89ee-10d8b8f18795 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:20.460794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13261d5a-4202-466e-89ee-10d8b8f18795 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:20.461102 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13261d5a-4202-466e-89ee-10d8b8f18795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:20.479438 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-13261d5a-4202-466e-89ee-10d8b8f18795 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:25:20.480652 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-13261d5a-4202-466e-89ee-10d8b8f18795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:20.488653 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1377/2767] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:25:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:25:20.492875 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f408e99-9af5-4491-8e83-e64894723b82 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:20.503279 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f408e99-9af5-4491-8e83-e64894723b82 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:20.508350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc101e-b03e-4f13-bd06-8a841976e4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:20.509075 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc101e-b03e-4f13-bd06-8a841976e4d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:20.509355 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc101e-b03e-4f13-bd06-8a841976e4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:20.509623 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc101e-b03e-4f13-bd06-8a841976e4d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:20.510381 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc101e-b03e-4f13-bd06-8a841976e4d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:25:20.510682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc101e-b03e-4f13-bd06-8a841976e4d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:20.512270 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc101e-b03e-4f13-bd06-8a841976e4d7 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:20.525358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1391/2768] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:25:20.529766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13261d5a-4202-466e-89ee-10d8b8f18795 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:20.539582 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13261d5a-4202-466e-89ee-10d8b8f18795 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:20.544386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0b2a8-9e47-4dce-a63a-514cf88ecee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:20.545107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0b2a8-9e47-4dce-a63a-514cf88ecee4 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:20.545392 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0b2a8-9e47-4dce-a63a-514cf88ecee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:20.545665 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec0b2a8-9e47-4dce-a63a-514cf88ecee4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:20.546435 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0b2a8-9e47-4dce-a63a-514cf88ecee4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:20.548352 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec0b2a8-9e47-4dce-a63a-514cf88ecee4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:20.588654 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1378/2769] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:25:20 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:25:20.594535 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc101e-b03e-4f13-bd06-8a841976e4d7 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:20.605086 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc101e-b03e-4f13-bd06-8a841976e4d7 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:20.650312 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:20.666707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5886ed-d811-4642-9a70-caeb90a3b8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:20.667400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5886ed-d811-4642-9a70-caeb90a3b8be admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:20.667682 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5886ed-d811-4642-9a70-caeb90a3b8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:20.668040 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5886ed-d811-4642-9a70-caeb90a3b8be admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:20.668800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5886ed-d811-4642-9a70-caeb90a3b8be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:20.670757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5886ed-d811-4642-9a70-caeb90a3b8be admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:20.695228 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1392/2770] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:25:20 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:25:20.699220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0b2a8-9e47-4dce-a63a-514cf88ecee4 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:20.706361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:20.710507 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec0b2a8-9e47-4dce-a63a-514cf88ecee4 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:20.716236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5e6d5-590c-455f-bc38-a79676cd109c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:20.717114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5e6d5-590c-455f-bc38-a79676cd109c admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:20.717440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5e6d5-590c-455f-bc38-a79676cd109c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:20.717878 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d5e6d5-590c-455f-bc38-a79676cd109c admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:20.718716 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d5e6d5-590c-455f-bc38-a79676cd109c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:20.720524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d5e6d5-590c-455f-bc38-a79676cd109c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:20.730556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1379/2771] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:25:20.734598 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5886ed-d811-4642-9a70-caeb90a3b8be admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:20.744461 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5886ed-d811-4642-9a70-caeb90a3b8be admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:20.750120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3614db-b495-4f76-bd17-bddeb4e9a173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:20.750776 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3614db-b495-4f76-bd17-bddeb4e9a173 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:20.751066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3614db-b495-4f76-bd17-bddeb4e9a173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:20.751341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3614db-b495-4f76-bd17-bddeb4e9a173 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:20.752071 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3614db-b495-4f76-bd17-bddeb4e9a173 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:20.753699 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3614db-b495-4f76-bd17-bddeb4e9a173 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:20.763838 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1393/2772] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:25:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:25:20.768412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5e6d5-590c-455f-bc38-a79676cd109c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:20.778517 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d5e6d5-590c-455f-bc38-a79676cd109c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:20.783254 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc354a-00e0-4db2-840d-2224fbf5d777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:20.784137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc354a-00e0-4db2-840d-2224fbf5d777 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:20.784416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc354a-00e0-4db2-840d-2224fbf5d777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:20.784686 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dc354a-00e0-4db2-840d-2224fbf5d777 admin admin] PATH_INFO: `/v3/projects/69a95f72de434efbb54ce04a51a8ddc3/users/a6001eb756ee40139bae27a0fb2c6d70/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:20.802069 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc354a-00e0-4db2-840d-2224fbf5d777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69a95f72de434efbb54ce04a51a8ddc3, user_id=a6001eb756ee40139bae27a0fb2c6d70, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:20.804708 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dc354a-00e0-4db2-840d-2224fbf5d777 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:20.818778 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1380/2773] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:25:20 2022] PUT /identity/v3/projects/69a95f72de434efbb54ce04a51a8ddc3/users/a6001eb756ee40139bae27a0fb2c6d70/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:20.829038 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd66e2-c32d-48a6-83e5-a8807c5f829c None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:20.829331 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd66e2-c32d-48a6-83e5-a8807c5f829c None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:20.829608 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd66e2-c32d-48a6-83e5-a8807c5f829c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:20.855960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-79bd66e2-c32d-48a6-83e5-a8807c5f829c None None] MFA Rules not processed for user `a6001eb756ee40139bae27a0fb2c6d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:25:20.857302 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-79bd66e2-c32d-48a6-83e5-a8807c5f829c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:20.921670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1394/2774] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:25:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:25:20.926258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc354a-00e0-4db2-840d-2224fbf5d777 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:20.989727 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dc354a-00e0-4db2-840d-2224fbf5d777 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:20.994246 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34a100-5c70-4b62-b18b-742e7f87e8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:20.994894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34a100-5c70-4b62-b18b-742e7f87e8d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:20.995176 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34a100-5c70-4b62-b18b-742e7f87e8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:20.995449 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e34a100-5c70-4b62-b18b-742e7f87e8d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:20.996146 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34a100-5c70-4b62-b18b-742e7f87e8d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:20.998157 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e34a100-5c70-4b62-b18b-742e7f87e8d0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:21.023644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1381/2775] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:25:20 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:25:21.027725 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd66e2-c32d-48a6-83e5-a8807c5f829c None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:21.038466 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd66e2-c32d-48a6-83e5-a8807c5f829c None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:21.125455 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef05527-0dbd-4b75-834b-82dfa38d2fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:21.126590 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef05527-0dbd-4b75-834b-82dfa38d2fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:21.127063 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef05527-0dbd-4b75-834b-82dfa38d2fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:21.127458 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef05527-0dbd-4b75-834b-82dfa38d2fc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:21.128653 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef05527-0dbd-4b75-834b-82dfa38d2fc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:21.132117 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef05527-0dbd-4b75-834b-82dfa38d2fc2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:21.155125 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1395/2776] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:25:21 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:25:21.159673 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34a100-5c70-4b62-b18b-742e7f87e8d0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:21.169938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e34a100-5c70-4b62-b18b-742e7f87e8d0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:21.175158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db439865-c134-4b85-b569-ee88832574f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:21.175812 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db439865-c134-4b85-b569-ee88832574f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:21.176137 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db439865-c134-4b85-b569-ee88832574f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:21.176398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db439865-c134-4b85-b569-ee88832574f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:21.177152 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db439865-c134-4b85-b569-ee88832574f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:21.178734 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db439865-c134-4b85-b569-ee88832574f0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:21.189114 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1382/2777] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:25:21.193285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef05527-0dbd-4b75-834b-82dfa38d2fc2 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:21.203350 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef05527-0dbd-4b75-834b-82dfa38d2fc2 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:21.208080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564dde54-d623-440d-bbee-c0c53d074865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:21.209031 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dde54-d623-440d-bbee-c0c53d074865 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:21.209312 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dde54-d623-440d-bbee-c0c53d074865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:21.209587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564dde54-d623-440d-bbee-c0c53d074865 admin admin] PATH_INFO: `/v3/projects/d7e13ee1ab5f4680b25d166c8bb35010/users/206fee9b242a469f80ee348c02982a2b/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:21.224468 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564dde54-d623-440d-bbee-c0c53d074865 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7e13ee1ab5f4680b25d166c8bb35010, user_id=206fee9b242a469f80ee348c02982a2b, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:21.227136 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564dde54-d623-440d-bbee-c0c53d074865 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:21.240785 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1396/2778] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:25:21 2022] PUT /identity/v3/projects/d7e13ee1ab5f4680b25d166c8bb35010/users/206fee9b242a469f80ee348c02982a2b/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:25:21.244834 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db439865-c134-4b85-b569-ee88832574f0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:21.255481 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db439865-c134-4b85-b569-ee88832574f0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:21.317826 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad7b8c-9da9-4882-b1a9-20387fb6e639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:21.318610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad7b8c-9da9-4882-b1a9-20387fb6e639 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:21.318938 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad7b8c-9da9-4882-b1a9-20387fb6e639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:21.319295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad7b8c-9da9-4882-b1a9-20387fb6e639 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:21.320179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad7b8c-9da9-4882-b1a9-20387fb6e639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:21.322147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad7b8c-9da9-4882-b1a9-20387fb6e639 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:21.335070 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1383/2779] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:25:21.339509 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564dde54-d623-440d-bbee-c0c53d074865 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:21.350664 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564dde54-d623-440d-bbee-c0c53d074865 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:21.355334 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c16dc-9cf2-4767-877b-b6f5adfa1111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:21.356221 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c16dc-9cf2-4767-877b-b6f5adfa1111 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:21.356503 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c16dc-9cf2-4767-877b-b6f5adfa1111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:21.356780 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c16dc-9cf2-4767-877b-b6f5adfa1111 admin admin] PATH_INFO: `/v3/projects/d7e13ee1ab5f4680b25d166c8bb35010/users/206fee9b242a469f80ee348c02982a2b/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:21.359340 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c16dc-9cf2-4767-877b-b6f5adfa1111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7e13ee1ab5f4680b25d166c8bb35010, user_id=206fee9b242a469f80ee348c02982a2b, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:21.362018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c16dc-9cf2-4767-877b-b6f5adfa1111 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:21.372726 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1397/2780] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:25:21 2022] PUT /identity/v3/projects/d7e13ee1ab5f4680b25d166c8bb35010/users/206fee9b242a469f80ee348c02982a2b/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:21.384574 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d49cb9-431a-443f-92dd-71f473b38396 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:21.384862 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d49cb9-431a-443f-92dd-71f473b38396 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:21.385184 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d49cb9-431a-443f-92dd-71f473b38396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:21.411606 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-17d49cb9-431a-443f-92dd-71f473b38396 None None] MFA Rules not processed for user `206fee9b242a469f80ee348c02982a2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:25:21.412817 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-17d49cb9-431a-443f-92dd-71f473b38396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:21.441663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:21.481622 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1384/2781] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:25:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:25:21.492441 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c16dc-9cf2-4767-877b-b6f5adfa1111 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:21.494080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-548c16dc-9cf2-4767-877b-b6f5adfa1111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:21.504153 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:21.556209 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c16dc-9cf2-4767-877b-b6f5adfa1111 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:21.560060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f1e1c-5dd7-4865-adae-bbe2596b8408 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:21.560716 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f1e1c-5dd7-4865-adae-bbe2596b8408 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:21.561030 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f1e1c-5dd7-4865-adae-bbe2596b8408 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:21.561310 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02f1e1c-5dd7-4865-adae-bbe2596b8408 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:21.618327 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f1e1c-5dd7-4865-adae-bbe2596b8408 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:21.619747 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02f1e1c-5dd7-4865-adae-bbe2596b8408 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:21.629173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1398/2782] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:25:21 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:25:26.241744 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d49cb9-431a-443f-92dd-71f473b38396 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:26.251643 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d49cb9-431a-443f-92dd-71f473b38396 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:26.255183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f67a6e-4b12-4c5f-aaeb-73a98b4cb1dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:26.255825 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f67a6e-4b12-4c5f-aaeb-73a98b4cb1dd service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:26.256107 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f67a6e-4b12-4c5f-aaeb-73a98b4cb1dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:26.256382 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f67a6e-4b12-4c5f-aaeb-73a98b4cb1dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:26.266420 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f67a6e-4b12-4c5f-aaeb-73a98b4cb1dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:26.267876 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f67a6e-4b12-4c5f-aaeb-73a98b4cb1dd service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:26.277262 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1385/2783] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:25:26 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:25:31.624769 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=45 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:36.273153 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=22 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:48.979935 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f1e1c-5dd7-4865-adae-bbe2596b8408 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:49.041554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02f1e1c-5dd7-4865-adae-bbe2596b8408 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:49.046114 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659fd1f-a297-4912-ab6f-c8d7d88e7176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:49.046988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659fd1f-a297-4912-ab6f-c8d7d88e7176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:49.047271 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659fd1f-a297-4912-ab6f-c8d7d88e7176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:49.047546 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c659fd1f-a297-4912-ab6f-c8d7d88e7176 admin admin] PATH_INFO: `/v3/users/a6001eb756ee40139bae27a0fb2c6d70` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:49.048758 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659fd1f-a297-4912-ab6f-c8d7d88e7176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6001eb756ee40139bae27a0fb2c6d70)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:49.050707 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c659fd1f-a297-4912-ab6f-c8d7d88e7176 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:49.097048 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c659fd1f-a297-4912-ab6f-c8d7d88e7176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6001eb756ee40139bae27a0fb2c6d70'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:49.097689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c659fd1f-a297-4912-ab6f-c8d7d88e7176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6001eb756ee40139bae27a0fb2c6d70'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:49.109171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c659fd1f-a297-4912-ab6f-c8d7d88e7176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6001eb756ee40139bae27a0fb2c6d70'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:49.116101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c659fd1f-a297-4912-ab6f-c8d7d88e7176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6001eb756ee40139bae27a0fb2c6d70'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:49.123589 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-c659fd1f-a297-4912-ab6f-c8d7d88e7176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6001eb756ee40139bae27a0fb2c6d70'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:49.129538 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1399/2784] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:25:48 2022] DELETE /identity/v3/users/a6001eb756ee40139bae27a0fb2c6d70 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:49.134281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a2194-bf1f-4976-88e1-173c35a02900 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:49.134791 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a2194-bf1f-4976-88e1-173c35a02900 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:49.134896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a2194-bf1f-4976-88e1-173c35a02900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:49.152461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-ae5a2194-bf1f-4976-88e1-173c35a02900 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:25:49.153694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ae5a2194-bf1f-4976-88e1-173c35a02900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:49.211315 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1386/2785] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:25:49.220955 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659fd1f-a297-4912-ab6f-c8d7d88e7176 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:49.222352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c659fd1f-a297-4912-ab6f-c8d7d88e7176 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:49.278587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c659fd1f-a297-4912-ab6f-c8d7d88e7176 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:49.282198 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeceef-64f7-4772-8a10-d348d073e162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:49.282836 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeceef-64f7-4772-8a10-d348d073e162 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:49.283111 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeceef-64f7-4772-8a10-d348d073e162 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:49.283381 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdeceef-64f7-4772-8a10-d348d073e162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:49.292073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdeceef-64f7-4772-8a10-d348d073e162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:49.293519 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdeceef-64f7-4772-8a10-d348d073e162 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:49.303183 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1400/2786] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:25:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:25:49.569398 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a2194-bf1f-4976-88e1-173c35a02900 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:49.571308 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-ae5a2194-bf1f-4976-88e1-173c35a02900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:49.586687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a2194-bf1f-4976-88e1-173c35a02900 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:49.591380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:49.592260 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:49.592672 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:49.592976 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b admin admin] PATH_INFO: `/v3/projects/69a95f72de434efbb54ce04a51a8ddc3` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:49.594153 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b admin admin] RBAC: Authorizing `identity:delete_project(project_id=69a95f72de434efbb54ce04a51a8ddc3)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:49.596116 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:49.641844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69a95f72de434efbb54ce04a51a8ddc3', 'request_id': 'req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:49.647911 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69a95f72de434efbb54ce04a51a8ddc3', 'request_id': 'req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:49.655688 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69a95f72de434efbb54ce04a51a8ddc3', 'request_id': 'req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:49.665888 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b admin admin] The token cache is being invalidate because project 69a95f72de434efbb54ce04a51a8ddc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:25:49.666165 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:49.666776 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:49.668293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1387/2787] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:25:49 2022] DELETE /identity/v3/projects/69a95f72de434efbb54ce04a51a8ddc3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:49.672807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeceef-64f7-4772-8a10-d348d073e162 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:49.682582 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdeceef-64f7-4772-8a10-d348d073e162 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:49.700451 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:49.740142 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94882fef-3a39-4d19-882c-e91accf79c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:49.741032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94882fef-3a39-4d19-882c-e91accf79c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:49.741314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94882fef-3a39-4d19-882c-e91accf79c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:49.741583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94882fef-3a39-4d19-882c-e91accf79c24 admin admin] PATH_INFO: `/v3/users/206fee9b242a469f80ee348c02982a2b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:49.742811 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94882fef-3a39-4d19-882c-e91accf79c24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=206fee9b242a469f80ee348c02982a2b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:49.744658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94882fef-3a39-4d19-882c-e91accf79c24 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:49.791257 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-94882fef-3a39-4d19-882c-e91accf79c24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '206fee9b242a469f80ee348c02982a2b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:49.791931 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-94882fef-3a39-4d19-882c-e91accf79c24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '206fee9b242a469f80ee348c02982a2b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:49.804422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-94882fef-3a39-4d19-882c-e91accf79c24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '206fee9b242a469f80ee348c02982a2b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:49.811217 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-94882fef-3a39-4d19-882c-e91accf79c24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '206fee9b242a469f80ee348c02982a2b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:49.818447 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-94882fef-3a39-4d19-882c-e91accf79c24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '206fee9b242a469f80ee348c02982a2b'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:49.824486 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1401/2788] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:25:49 2022] DELETE /identity/v3/users/206fee9b242a469f80ee348c02982a2b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:50.090529 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:50.092480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:50.109080 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ccf4b8-cfe9-4620-b3a4-efa53b1f160b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:50.166506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315fa42-3e5c-4fb7-9a5e-637d37d2afd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:50.167363 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315fa42-3e5c-4fb7-9a5e-637d37d2afd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:50.167644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315fa42-3e5c-4fb7-9a5e-637d37d2afd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:50.167918 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315fa42-3e5c-4fb7-9a5e-637d37d2afd9 admin admin] PATH_INFO: `/v3/projects/d7e13ee1ab5f4680b25d166c8bb35010` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:50.169121 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315fa42-3e5c-4fb7-9a5e-637d37d2afd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7e13ee1ab5f4680b25d166c8bb35010)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:50.171280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315fa42-3e5c-4fb7-9a5e-637d37d2afd9 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:50.200457 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:50.218763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2315fa42-3e5c-4fb7-9a5e-637d37d2afd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7e13ee1ab5f4680b25d166c8bb35010', 'request_id': 'req-2315fa42-3e5c-4fb7-9a5e-637d37d2afd9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:50.224631 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2315fa42-3e5c-4fb7-9a5e-637d37d2afd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7e13ee1ab5f4680b25d166c8bb35010', 'request_id': 'req-2315fa42-3e5c-4fb7-9a5e-637d37d2afd9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:50.231864 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2315fa42-3e5c-4fb7-9a5e-637d37d2afd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7e13ee1ab5f4680b25d166c8bb35010', 'request_id': 'req-2315fa42-3e5c-4fb7-9a5e-637d37d2afd9'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:50.242844 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2315fa42-3e5c-4fb7-9a5e-637d37d2afd9 admin admin] The token cache is being invalidate because project d7e13ee1ab5f4680b25d166c8bb35010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:25:50.243131 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2315fa42-3e5c-4fb7-9a5e-637d37d2afd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:50.243768 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-2315fa42-3e5c-4fb7-9a5e-637d37d2afd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:50.245450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1388/2789] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:25:50 2022] DELETE /identity/v3/projects/d7e13ee1ab5f4680b25d166c8bb35010 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:50.250592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f74f16-1629-4f1a-93bb-d7a1a7ae8ac3 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:50.251013 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f74f16-1629-4f1a-93bb-d7a1a7ae8ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:50.251297 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f74f16-1629-4f1a-93bb-d7a1a7ae8ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:50.270081 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-58f74f16-1629-4f1a-93bb-d7a1a7ae8ac3 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:25:50.271211 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-58f74f16-1629-4f1a-93bb-d7a1a7ae8ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:50.329641 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1402/2790] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:25:50.337492 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b9745-0db3-4bf2-939a-bd8a209412c4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:50.337784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b9745-0db3-4bf2-939a-bd8a209412c4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:50.338059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b9745-0db3-4bf2-939a-bd8a209412c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:50.356530 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-dc9b9745-0db3-4bf2-939a-bd8a209412c4 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:25:50.357813 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-dc9b9745-0db3-4bf2-939a-bd8a209412c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:50.366177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1389/2791] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:25:50.370393 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f74f16-1629-4f1a-93bb-d7a1a7ae8ac3 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:50.379943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f74f16-1629-4f1a-93bb-d7a1a7ae8ac3 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:50.384865 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1bddf8-6e2f-43ec-b38e-1656d52b2cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:50.385544 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bddf8-6e2f-43ec-b38e-1656d52b2cce admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:50.385825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bddf8-6e2f-43ec-b38e-1656d52b2cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:50.386099 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1bddf8-6e2f-43ec-b38e-1656d52b2cce admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:50.386827 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bddf8-6e2f-43ec-b38e-1656d52b2cce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:25:50.387130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bddf8-6e2f-43ec-b38e-1656d52b2cce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:50.389253 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1bddf8-6e2f-43ec-b38e-1656d52b2cce admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:50.400599 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1403/2792] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:25:50.405377 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b9745-0db3-4bf2-939a-bd8a209412c4 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:50.414549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b9745-0db3-4bf2-939a-bd8a209412c4 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:50.419711 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af170af7-a604-4346-b5bc-d4de32f57d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:50.420581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af170af7-a604-4346-b5bc-d4de32f57d44 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:50.421082 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af170af7-a604-4346-b5bc-d4de32f57d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:50.421508 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af170af7-a604-4346-b5bc-d4de32f57d44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:50.422388 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af170af7-a604-4346-b5bc-d4de32f57d44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:50.424852 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af170af7-a604-4346-b5bc-d4de32f57d44 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:50.451773 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1390/2793] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:25:50 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:25:50.456109 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1bddf8-6e2f-43ec-b38e-1656d52b2cce admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:50.466059 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1bddf8-6e2f-43ec-b38e-1656d52b2cce admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:50.492367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:50.524047 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb50f02-cf17-439f-aadd-beb505c3d7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:50.524665 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb50f02-cf17-439f-aadd-beb505c3d7d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:50.524978 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb50f02-cf17-439f-aadd-beb505c3d7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:50.525333 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb50f02-cf17-439f-aadd-beb505c3d7d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:50.526031 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb50f02-cf17-439f-aadd-beb505c3d7d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:50.527925 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb50f02-cf17-439f-aadd-beb505c3d7d8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:50.549171 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1404/2794] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:25:50 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:25:50.553258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af170af7-a604-4346-b5bc-d4de32f57d44 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:50.563393 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af170af7-a604-4346-b5bc-d4de32f57d44 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:50.567850 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87210e-6a99-4ccd-a181-eddb9e30a0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:50.568499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87210e-6a99-4ccd-a181-eddb9e30a0ea admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:50.568772 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87210e-6a99-4ccd-a181-eddb9e30a0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:50.569079 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a87210e-6a99-4ccd-a181-eddb9e30a0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:50.569801 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87210e-6a99-4ccd-a181-eddb9e30a0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:50.571365 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a87210e-6a99-4ccd-a181-eddb9e30a0ea admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:50.582506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1391/2795] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:25:50.586301 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb50f02-cf17-439f-aadd-beb505c3d7d8 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:50.596695 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb50f02-cf17-439f-aadd-beb505c3d7d8 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:50.601240 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6330a176-60eb-4d88-b159-e280d650ef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:50.601888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330a176-60eb-4d88-b159-e280d650ef02 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:50.602168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330a176-60eb-4d88-b159-e280d650ef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:50.602442 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6330a176-60eb-4d88-b159-e280d650ef02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:50.603168 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330a176-60eb-4d88-b159-e280d650ef02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:50.604825 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6330a176-60eb-4d88-b159-e280d650ef02 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:50.615085 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1405/2796] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:25:50.619023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87210e-6a99-4ccd-a181-eddb9e30a0ea admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:50.628945 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a87210e-6a99-4ccd-a181-eddb9e30a0ea admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:50.633964 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72a763-53e6-4071-8ee0-d00218266a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:50.634827 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72a763-53e6-4071-8ee0-d00218266a11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:50.635151 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72a763-53e6-4071-8ee0-d00218266a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:50.635400 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc72a763-53e6-4071-8ee0-d00218266a11 admin admin] PATH_INFO: `/v3/projects/7a58332ce6c94873ba3d02026bf2eae4/users/b35d2a49615746dca913e511f7f9354b/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:50.649785 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72a763-53e6-4071-8ee0-d00218266a11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a58332ce6c94873ba3d02026bf2eae4, user_id=b35d2a49615746dca913e511f7f9354b, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:50.652364 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc72a763-53e6-4071-8ee0-d00218266a11 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:50.665191 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1392/2797] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:25:50 2022] PUT /identity/v3/projects/7a58332ce6c94873ba3d02026bf2eae4/users/b35d2a49615746dca913e511f7f9354b/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:50.676005 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fee0e-7db4-4190-adeb-e8a6ac3a8c2a None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:50.676298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fee0e-7db4-4190-adeb-e8a6ac3a8c2a None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:50.676594 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9fee0e-7db4-4190-adeb-e8a6ac3a8c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:50.702201 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-2e9fee0e-7db4-4190-adeb-e8a6ac3a8c2a None None] MFA Rules not processed for user `b35d2a49615746dca913e511f7f9354b`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:25:50.703404 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-2e9fee0e-7db4-4190-adeb-e8a6ac3a8c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:50.768236 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1406/2798] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:25:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:25:50.772707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72a763-53e6-4071-8ee0-d00218266a11 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:50.833670 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc72a763-53e6-4071-8ee0-d00218266a11 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:50.838237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fc5e6-dd08-4e63-a8de-20f3d8468102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:50.838893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fc5e6-dd08-4e63-a8de-20f3d8468102 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:50.839176 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fc5e6-dd08-4e63-a8de-20f3d8468102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:50.839450 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1fc5e6-dd08-4e63-a8de-20f3d8468102 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:50.840148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fc5e6-dd08-4e63-a8de-20f3d8468102 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:50.842103 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1fc5e6-dd08-4e63-a8de-20f3d8468102 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:50.866953 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1393/2799] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:25:50 2022] POST /identity/v3/projects => generated 379 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:25:50.871029 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9fee0e-7db4-4190-adeb-e8a6ac3a8c2a None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:50.883369 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9fee0e-7db4-4190-adeb-e8a6ac3a8c2a None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:50.938927 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c3e4e-41f4-420b-901a-0410adf3d4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:50.939591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c3e4e-41f4-420b-901a-0410adf3d4f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:50.939871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c3e4e-41f4-420b-901a-0410adf3d4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:50.940143 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406c3e4e-41f4-420b-901a-0410adf3d4f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:50.940835 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c3e4e-41f4-420b-901a-0410adf3d4f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:50.942871 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406c3e4e-41f4-420b-901a-0410adf3d4f4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:50.962118 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1407/2800] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:25:50 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:25:50.965966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fc5e6-dd08-4e63-a8de-20f3d8468102 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:50.976506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1fc5e6-dd08-4e63-a8de-20f3d8468102 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:50.981029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8581418-deae-4a6e-8267-2c2b27b840f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:50.981683 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8581418-deae-4a6e-8267-2c2b27b840f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:50.982092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8581418-deae-4a6e-8267-2c2b27b840f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:50.982373 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8581418-deae-4a6e-8267-2c2b27b840f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:50.983104 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8581418-deae-4a6e-8267-2c2b27b840f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:50.984808 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8581418-deae-4a6e-8267-2c2b27b840f7 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:51.001902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1394/2801] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:25:51.006739 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c3e4e-41f4-420b-901a-0410adf3d4f4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:51.017895 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406c3e4e-41f4-420b-901a-0410adf3d4f4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:51.023660 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697bbbd-923a-48d6-ad58-35e273a50f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:51.024556 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697bbbd-923a-48d6-ad58-35e273a50f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:51.024984 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697bbbd-923a-48d6-ad58-35e273a50f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:51.025373 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e697bbbd-923a-48d6-ad58-35e273a50f76 admin admin] PATH_INFO: `/v3/projects/7722f7608e014249bb050bc5fdbf89c8/users/13aae08e7f2d4f4788632f793260c7ff/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:51.041464 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697bbbd-923a-48d6-ad58-35e273a50f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7722f7608e014249bb050bc5fdbf89c8, user_id=13aae08e7f2d4f4788632f793260c7ff, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:51.044067 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e697bbbd-923a-48d6-ad58-35e273a50f76 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:51.056712 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1408/2802] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:25:51 2022] PUT /identity/v3/projects/7722f7608e014249bb050bc5fdbf89c8/users/13aae08e7f2d4f4788632f793260c7ff/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:51.060740 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8581418-deae-4a6e-8267-2c2b27b840f7 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:51.070639 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8581418-deae-4a6e-8267-2c2b27b840f7 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:51.091896 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:51.130204 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f9d1bb-d00e-4464-ba78-3a43066ca9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:51.130872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9d1bb-d00e-4464-ba78-3a43066ca9ae admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:51.131179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9d1bb-d00e-4464-ba78-3a43066ca9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:51.131710 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f9d1bb-d00e-4464-ba78-3a43066ca9ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:51.132433 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9d1bb-d00e-4464-ba78-3a43066ca9ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:51.134061 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f9d1bb-d00e-4464-ba78-3a43066ca9ae admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:51.144129 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1395/2803] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:25:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:25:51.148303 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697bbbd-923a-48d6-ad58-35e273a50f76 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:51.153782 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:51.160814 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e697bbbd-923a-48d6-ad58-35e273a50f76 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:51.165800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37abfed-62c5-4ae8-940a-ecf6d07bcdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:51.166738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37abfed-62c5-4ae8-940a-ecf6d07bcdaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:51.167014 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37abfed-62c5-4ae8-940a-ecf6d07bcdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:51.167285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37abfed-62c5-4ae8-940a-ecf6d07bcdaf admin admin] PATH_INFO: `/v3/projects/7722f7608e014249bb050bc5fdbf89c8/users/13aae08e7f2d4f4788632f793260c7ff/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:51.169345 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37abfed-62c5-4ae8-940a-ecf6d07bcdaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7722f7608e014249bb050bc5fdbf89c8, user_id=13aae08e7f2d4f4788632f793260c7ff, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:51.171998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37abfed-62c5-4ae8-940a-ecf6d07bcdaf admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:51.182226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1409/2804] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:25:51 2022] PUT /identity/v3/projects/7722f7608e014249bb050bc5fdbf89c8/users/13aae08e7f2d4f4788632f793260c7ff/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:51.193827 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f7854-601a-4710-aaf8-af0cc23ca557 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:51.194303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f7854-601a-4710-aaf8-af0cc23ca557 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:51.194403 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f7854-601a-4710-aaf8-af0cc23ca557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:51.220751 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-e44f7854-601a-4710-aaf8-af0cc23ca557 None None] MFA Rules not processed for user `13aae08e7f2d4f4788632f793260c7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:25:51.222076 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e44f7854-601a-4710-aaf8-af0cc23ca557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:51.288728 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1396/2805] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:25:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:25:51.299597 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37abfed-62c5-4ae8-940a-ecf6d07bcdaf admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:51.301412 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c37abfed-62c5-4ae8-940a-ecf6d07bcdaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:51.357645 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37abfed-62c5-4ae8-940a-ecf6d07bcdaf admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:51.361189 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf8784-98a4-47a4-90f2-8731bfe1ed73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:51.361828 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf8784-98a4-47a4-90f2-8731bfe1ed73 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:51.362111 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf8784-98a4-47a4-90f2-8731bfe1ed73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:51.362383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf8784-98a4-47a4-90f2-8731bfe1ed73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:51.371249 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf8784-98a4-47a4-90f2-8731bfe1ed73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:51.373281 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf8784-98a4-47a4-90f2-8731bfe1ed73 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:51.382706 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1410/2806] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:25:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:25:51.765885 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f7854-601a-4710-aaf8-af0cc23ca557 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:51.829625 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f7854-601a-4710-aaf8-af0cc23ca557 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:51.834169 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96e645-aabc-475d-85dc-50b9ea3aa88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:51.835038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96e645-aabc-475d-85dc-50b9ea3aa88d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:51.835317 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96e645-aabc-475d-85dc-50b9ea3aa88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:51.835589 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f96e645-aabc-475d-85dc-50b9ea3aa88d admin admin] PATH_INFO: `/v3/users/b35d2a49615746dca913e511f7f9354b` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:51.836801 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96e645-aabc-475d-85dc-50b9ea3aa88d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b35d2a49615746dca913e511f7f9354b)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:51.838750 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f96e645-aabc-475d-85dc-50b9ea3aa88d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:51.885475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5f96e645-aabc-475d-85dc-50b9ea3aa88d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b35d2a49615746dca913e511f7f9354b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:51.886043 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5f96e645-aabc-475d-85dc-50b9ea3aa88d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b35d2a49615746dca913e511f7f9354b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:51.887198 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:51.900284 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5f96e645-aabc-475d-85dc-50b9ea3aa88d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b35d2a49615746dca913e511f7f9354b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:51.907638 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5f96e645-aabc-475d-85dc-50b9ea3aa88d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b35d2a49615746dca913e511f7f9354b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:51.914745 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-5f96e645-aabc-475d-85dc-50b9ea3aa88d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b35d2a49615746dca913e511f7f9354b'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:51.921276 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1397/2807] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:25:51 2022] DELETE /identity/v3/users/b35d2a49615746dca913e511f7f9354b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:51.925796 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74afc3-2e4f-4ba9-8386-60374ab1a4dc None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:51.926101 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74afc3-2e4f-4ba9-8386-60374ab1a4dc None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:51.926420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee74afc3-2e4f-4ba9-8386-60374ab1a4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:51.943443 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ee74afc3-2e4f-4ba9-8386-60374ab1a4dc None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:25:51.944657 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ee74afc3-2e4f-4ba9-8386-60374ab1a4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:52.001414 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1411/2808] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:25:52.011280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96e645-aabc-475d-85dc-50b9ea3aa88d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:52.012644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-5f96e645-aabc-475d-85dc-50b9ea3aa88d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:52.067765 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f96e645-aabc-475d-85dc-50b9ea3aa88d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:52.071332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73020032-16f9-4c6c-a5cd-db693223213f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:52.071965 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73020032-16f9-4c6c-a5cd-db693223213f service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:52.072242 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73020032-16f9-4c6c-a5cd-db693223213f service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:52.072513 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73020032-16f9-4c6c-a5cd-db693223213f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:52.081397 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73020032-16f9-4c6c-a5cd-db693223213f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:52.082837 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73020032-16f9-4c6c-a5cd-db693223213f service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:52.092591 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1398/2809] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:25:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:25:52.381979 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74afc3-2e4f-4ba9-8386-60374ab1a4dc None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:52.383471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ee74afc3-2e4f-4ba9-8386-60374ab1a4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:52.396728 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee74afc3-2e4f-4ba9-8386-60374ab1a4dc None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:52.401264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcfa115-1321-499b-8610-1bc2e0a662db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:52.402137 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfa115-1321-499b-8610-1bc2e0a662db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:52.402420 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfa115-1321-499b-8610-1bc2e0a662db admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:52.402695 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcfa115-1321-499b-8610-1bc2e0a662db admin admin] PATH_INFO: `/v3/projects/7a58332ce6c94873ba3d02026bf2eae4` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:52.404489 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcfa115-1321-499b-8610-1bc2e0a662db admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a58332ce6c94873ba3d02026bf2eae4)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:52.406550 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcfa115-1321-499b-8610-1bc2e0a662db admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:52.457358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fdcfa115-1321-499b-8610-1bc2e0a662db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a58332ce6c94873ba3d02026bf2eae4', 'request_id': 'req-fdcfa115-1321-499b-8610-1bc2e0a662db'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:52.463940 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fdcfa115-1321-499b-8610-1bc2e0a662db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a58332ce6c94873ba3d02026bf2eae4', 'request_id': 'req-fdcfa115-1321-499b-8610-1bc2e0a662db'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:52.469551 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fdcfa115-1321-499b-8610-1bc2e0a662db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a58332ce6c94873ba3d02026bf2eae4', 'request_id': 'req-fdcfa115-1321-499b-8610-1bc2e0a662db'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:52.482460 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fdcfa115-1321-499b-8610-1bc2e0a662db admin admin] The token cache is being invalidate because project 7a58332ce6c94873ba3d02026bf2eae4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:25:52.482743 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fdcfa115-1321-499b-8610-1bc2e0a662db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:52.483435 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-fdcfa115-1321-499b-8610-1bc2e0a662db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:52.485035 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1412/2810] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:25:52 2022] DELETE /identity/v3/projects/7a58332ce6c94873ba3d02026bf2eae4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:52.489003 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73020032-16f9-4c6c-a5cd-db693223213f service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:52.499281 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73020032-16f9-4c6c-a5cd-db693223213f service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:52.553760 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19ac37d-6fa6-4d8e-9d17-06fe37b133ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:52.554624 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ac37d-6fa6-4d8e-9d17-06fe37b133ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:52.554908 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ac37d-6fa6-4d8e-9d17-06fe37b133ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:52.555212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19ac37d-6fa6-4d8e-9d17-06fe37b133ba admin admin] PATH_INFO: `/v3/users/13aae08e7f2d4f4788632f793260c7ff` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:52.556401 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ac37d-6fa6-4d8e-9d17-06fe37b133ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=13aae08e7f2d4f4788632f793260c7ff)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:52.558350 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19ac37d-6fa6-4d8e-9d17-06fe37b133ba admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:52.604473 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e19ac37d-6fa6-4d8e-9d17-06fe37b133ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13aae08e7f2d4f4788632f793260c7ff'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:52.605158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e19ac37d-6fa6-4d8e-9d17-06fe37b133ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13aae08e7f2d4f4788632f793260c7ff'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:52.617304 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e19ac37d-6fa6-4d8e-9d17-06fe37b133ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13aae08e7f2d4f4788632f793260c7ff'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:52.621893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:52.627523 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e19ac37d-6fa6-4d8e-9d17-06fe37b133ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13aae08e7f2d4f4788632f793260c7ff'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:52.635971 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e19ac37d-6fa6-4d8e-9d17-06fe37b133ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13aae08e7f2d4f4788632f793260c7ff'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:52.642127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1399/2811] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:25:52 2022] DELETE /identity/v3/users/13aae08e7f2d4f4788632f793260c7ff => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:52.916400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcfa115-1321-499b-8610-1bc2e0a662db admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:52.917847 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-fdcfa115-1321-499b-8610-1bc2e0a662db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:52.931745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcfa115-1321-499b-8610-1bc2e0a662db admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:52.944113 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:52.990734 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212fd69-ff6d-49c8-ae17-6d27b6a8c861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:52.991519 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212fd69-ff6d-49c8-ae17-6d27b6a8c861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:52.991800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212fd69-ff6d-49c8-ae17-6d27b6a8c861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:52.992072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212fd69-ff6d-49c8-ae17-6d27b6a8c861 admin admin] PATH_INFO: `/v3/projects/7722f7608e014249bb050bc5fdbf89c8` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:52.993294 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212fd69-ff6d-49c8-ae17-6d27b6a8c861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7722f7608e014249bb050bc5fdbf89c8)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:52.995188 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212fd69-ff6d-49c8-ae17-6d27b6a8c861 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:53.040370 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5212fd69-ff6d-49c8-ae17-6d27b6a8c861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7722f7608e014249bb050bc5fdbf89c8', 'request_id': 'req-5212fd69-ff6d-49c8-ae17-6d27b6a8c861'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:53.046340 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5212fd69-ff6d-49c8-ae17-6d27b6a8c861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7722f7608e014249bb050bc5fdbf89c8', 'request_id': 'req-5212fd69-ff6d-49c8-ae17-6d27b6a8c861'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:53.051718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5212fd69-ff6d-49c8-ae17-6d27b6a8c861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7722f7608e014249bb050bc5fdbf89c8', 'request_id': 'req-5212fd69-ff6d-49c8-ae17-6d27b6a8c861'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:53.063737 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5212fd69-ff6d-49c8-ae17-6d27b6a8c861 admin admin] The token cache is being invalidate because project 7722f7608e014249bb050bc5fdbf89c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:25:53.064025 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5212fd69-ff6d-49c8-ae17-6d27b6a8c861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:53.064667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-5212fd69-ff6d-49c8-ae17-6d27b6a8c861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:25:53.066214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1413/2812] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:25:52 2022] DELETE /identity/v3/projects/7722f7608e014249bb050bc5fdbf89c8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:53.071207 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c210d6-1ba4-406e-b604-98ec98ab7c98 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:53.071521 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c210d6-1ba4-406e-b604-98ec98ab7c98 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:53.071802 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c210d6-1ba4-406e-b604-98ec98ab7c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:53.093244 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-86c210d6-1ba4-406e-b604-98ec98ab7c98 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:25:53.094463 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-86c210d6-1ba4-406e-b604-98ec98ab7c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:53.154539 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1400/2813] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:25:53.162378 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c73639-562d-4fad-8710-7c5703a60009 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:53.162670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c73639-562d-4fad-8710-7c5703a60009 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:53.162942 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c73639-562d-4fad-8710-7c5703a60009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:53.181741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-c7c73639-562d-4fad-8710-7c5703a60009 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:25:53.182942 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-c7c73639-562d-4fad-8710-7c5703a60009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:53.190822 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1414/2814] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:25:53.195059 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c210d6-1ba4-406e-b604-98ec98ab7c98 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:53.205428 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c210d6-1ba4-406e-b604-98ec98ab7c98 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:53.210037 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cc5f1-273b-40af-b891-d8167b33c4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:53.210943 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cc5f1-273b-40af-b891-d8167b33c4db admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:53.211231 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cc5f1-273b-40af-b891-d8167b33c4db admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:53.211503 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9cc5f1-273b-40af-b891-d8167b33c4db admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:53.212229 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cc5f1-273b-40af-b891-d8167b33c4db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:25:53.212536 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cc5f1-273b-40af-b891-d8167b33c4db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:53.214159 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9cc5f1-273b-40af-b891-d8167b33c4db admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:53.226032 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1401/2815] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 14 10:25:53.230305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c73639-562d-4fad-8710-7c5703a60009 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:53.239517 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c73639-562d-4fad-8710-7c5703a60009 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:53.244161 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc71d7-dfb1-4933-9000-174296dbdcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:53.244757 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc71d7-dfb1-4933-9000-174296dbdcd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:53.245073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc71d7-dfb1-4933-9000-174296dbdcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:53.245348 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc71d7-dfb1-4933-9000-174296dbdcd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:53.246066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc71d7-dfb1-4933-9000-174296dbdcd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:53.248089 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc71d7-dfb1-4933-9000-174296dbdcd2 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:53.273689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1415/2816] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:25:53 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:25:53.277887 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cc5f1-273b-40af-b891-d8167b33c4db admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:53.288511 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9cc5f1-273b-40af-b891-d8167b33c4db admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:53.342644 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544e06d-fb0b-4aa4-8a59-6ff52136a9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:53.343296 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544e06d-fb0b-4aa4-8a59-6ff52136a9d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:53.343578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544e06d-fb0b-4aa4-8a59-6ff52136a9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:53.343853 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544e06d-fb0b-4aa4-8a59-6ff52136a9d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:53.344545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544e06d-fb0b-4aa4-8a59-6ff52136a9d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:53.346483 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544e06d-fb0b-4aa4-8a59-6ff52136a9d6 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:53.365647 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1402/2817] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:25:53 2022] POST /identity/v3/users => generated 289 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:25:53.369840 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc71d7-dfb1-4933-9000-174296dbdcd2 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:53.380314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc71d7-dfb1-4933-9000-174296dbdcd2 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:53.385154 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac5782-8030-4b33-a7d9-244814373c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:53.385806 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac5782-8030-4b33-a7d9-244814373c06 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:53.386085 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac5782-8030-4b33-a7d9-244814373c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:53.386359 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fac5782-8030-4b33-a7d9-244814373c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:53.387078 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac5782-8030-4b33-a7d9-244814373c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:53.389184 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fac5782-8030-4b33-a7d9-244814373c06 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:53.399417 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1416/2818] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:25:53.403211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544e06d-fb0b-4aa4-8a59-6ff52136a9d6 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:53.413418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544e06d-fb0b-4aa4-8a59-6ff52136a9d6 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:53.418109 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb54990-e858-4e96-9f98-98f23f6599c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:53.418759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb54990-e858-4e96-9f98-98f23f6599c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:53.419040 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb54990-e858-4e96-9f98-98f23f6599c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:53.419338 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb54990-e858-4e96-9f98-98f23f6599c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:53.420062 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb54990-e858-4e96-9f98-98f23f6599c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:53.421661 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb54990-e858-4e96-9f98-98f23f6599c5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:53.431418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1403/2819] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:25:53.435945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac5782-8030-4b33-a7d9-244814373c06 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:53.445622 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fac5782-8030-4b33-a7d9-244814373c06 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:53.450143 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f65a90-d54d-476b-b114-f93017dc41a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:53.451587 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f65a90-d54d-476b-b114-f93017dc41a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:53.451858 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f65a90-d54d-476b-b114-f93017dc41a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:53.452125 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f65a90-d54d-476b-b114-f93017dc41a6 admin admin] PATH_INFO: `/v3/projects/af3c38819c5e4d309f60cfe0946e42ec/users/af6aaf545c0c4e6a951cae8be883f7a8/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:53.466490 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f65a90-d54d-476b-b114-f93017dc41a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af3c38819c5e4d309f60cfe0946e42ec, user_id=af6aaf545c0c4e6a951cae8be883f7a8, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:53.469178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f65a90-d54d-476b-b114-f93017dc41a6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:53.481715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1417/2820] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:25:53 2022] PUT /identity/v3/projects/af3c38819c5e4d309f60cfe0946e42ec/users/af6aaf545c0c4e6a951cae8be883f7a8/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:25:53.542944 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd54a7-5754-4dbb-8213-0cb926513547 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:53.543509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd54a7-5754-4dbb-8213-0cb926513547 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:53.543509 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fd54a7-5754-4dbb-8213-0cb926513547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:53.570558 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-f3fd54a7-5754-4dbb-8213-0cb926513547 None None] MFA Rules not processed for user `af6aaf545c0c4e6a951cae8be883f7a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:25:53.571766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-f3fd54a7-5754-4dbb-8213-0cb926513547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:53.612002 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:53.636077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1404/2821] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:25:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:25:53.647343 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f65a90-d54d-476b-b114-f93017dc41a6 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:25:53.649030 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-63f65a90-d54d-476b-b114-f93017dc41a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:25:53.696076 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:25:53.730200 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f65a90-d54d-476b-b114-f93017dc41a6 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:25:53.734976 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881615b3-dbd9-47f1-a22d-912fe1d41048 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e3b5660cfd344dcd8abe14c88df686e8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:25:53.735623 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881615b3-dbd9-47f1-a22d-912fe1d41048 service glance] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:25:53.735905 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881615b3-dbd9-47f1-a22d-912fe1d41048 service glance] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:25:53.736179 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881615b3-dbd9-47f1-a22d-912fe1d41048 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:25:53.744308 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881615b3-dbd9-47f1-a22d-912fe1d41048 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:25:53.746049 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881615b3-dbd9-47f1-a22d-912fe1d41048 service glance] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:25:53.754845 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1418/2822] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:25:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:26:03.633189 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=8 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:03.750748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=16 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:09.165290 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd54a7-5754-4dbb-8213-0cb926513547 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:09.225451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fd54a7-5754-4dbb-8213-0cb926513547 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:09.230106 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde0764-b09b-4e20-b605-c04f1a6ff5b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:09.231517 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde0764-b09b-4e20-b605-c04f1a6ff5b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:09.231795 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde0764-b09b-4e20-b605-c04f1a6ff5b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:09.232061 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfde0764-b09b-4e20-b605-c04f1a6ff5b0 admin admin] PATH_INFO: `/v3/users/af6aaf545c0c4e6a951cae8be883f7a8` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:09.233258 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde0764-b09b-4e20-b605-c04f1a6ff5b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af6aaf545c0c4e6a951cae8be883f7a8)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:09.235188 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfde0764-b09b-4e20-b605-c04f1a6ff5b0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:09.281381 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cfde0764-b09b-4e20-b605-c04f1a6ff5b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af6aaf545c0c4e6a951cae8be883f7a8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:09.282029 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cfde0764-b09b-4e20-b605-c04f1a6ff5b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af6aaf545c0c4e6a951cae8be883f7a8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:09.294303 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cfde0764-b09b-4e20-b605-c04f1a6ff5b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af6aaf545c0c4e6a951cae8be883f7a8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:09.301531 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cfde0764-b09b-4e20-b605-c04f1a6ff5b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af6aaf545c0c4e6a951cae8be883f7a8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:09.308722 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-cfde0764-b09b-4e20-b605-c04f1a6ff5b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af6aaf545c0c4e6a951cae8be883f7a8'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:09.314657 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1405/2823] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:26:09 2022] DELETE /identity/v3/users/af6aaf545c0c4e6a951cae8be883f7a8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:09.319166 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d42f19-1479-4b08-9a7b-437abebc866b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:09.319673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d42f19-1479-4b08-9a7b-437abebc866b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:09.319784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d42f19-1479-4b08-9a7b-437abebc866b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:09.337060 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-92d42f19-1479-4b08-9a7b-437abebc866b None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:09.338266 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-92d42f19-1479-4b08-9a7b-437abebc866b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:09.394945 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1419/2824] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:26:09.404995 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde0764-b09b-4e20-b605-c04f1a6ff5b0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:09.406370 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-cfde0764-b09b-4e20-b605-c04f1a6ff5b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:09.461150 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfde0764-b09b-4e20-b605-c04f1a6ff5b0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:09.464685 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6d3f98-02bb-47d6-b214-b7ab6e980f15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:09.465839 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d3f98-02bb-47d6-b214-b7ab6e980f15 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:09.466113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d3f98-02bb-47d6-b214-b7ab6e980f15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:09.466382 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d3f98-02bb-47d6-b214-b7ab6e980f15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:09.475156 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d3f98-02bb-47d6-b214-b7ab6e980f15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:09.476561 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6d3f98-02bb-47d6-b214-b7ab6e980f15 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:09.486280 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1406/2825] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:26:09.759104 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d42f19-1479-4b08-9a7b-437abebc866b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:09.760505 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-92d42f19-1479-4b08-9a7b-437abebc866b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:09.775325 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d42f19-1479-4b08-9a7b-437abebc866b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:09.781238 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d088cc-c2cd-432e-9ff8-95390dee49b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:09.782452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d088cc-c2cd-432e-9ff8-95390dee49b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:09.782718 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d088cc-c2cd-432e-9ff8-95390dee49b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:09.782992 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d088cc-c2cd-432e-9ff8-95390dee49b6 admin admin] PATH_INFO: `/v3/projects/af3c38819c5e4d309f60cfe0946e42ec` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:09.784178 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d088cc-c2cd-432e-9ff8-95390dee49b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af3c38819c5e4d309f60cfe0946e42ec)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:09.786102 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d088cc-c2cd-432e-9ff8-95390dee49b6 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:09.831754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-52d088cc-c2cd-432e-9ff8-95390dee49b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af3c38819c5e4d309f60cfe0946e42ec', 'request_id': 'req-52d088cc-c2cd-432e-9ff8-95390dee49b6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:09.837643 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-52d088cc-c2cd-432e-9ff8-95390dee49b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af3c38819c5e4d309f60cfe0946e42ec', 'request_id': 'req-52d088cc-c2cd-432e-9ff8-95390dee49b6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:09.843130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-52d088cc-c2cd-432e-9ff8-95390dee49b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af3c38819c5e4d309f60cfe0946e42ec', 'request_id': 'req-52d088cc-c2cd-432e-9ff8-95390dee49b6'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:09.854915 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-52d088cc-c2cd-432e-9ff8-95390dee49b6 admin admin] The token cache is being invalidate because project af3c38819c5e4d309f60cfe0946e42ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:26:09.855199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-52d088cc-c2cd-432e-9ff8-95390dee49b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:09.856351 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-52d088cc-c2cd-432e-9ff8-95390dee49b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:09.858032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1420/2826] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:26:09 2022] DELETE /identity/v3/projects/af3c38819c5e4d309f60cfe0946e42ec => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:09.863028 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3862e-e565-429f-83da-43ca2180fa57 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:09.863336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3862e-e565-429f-83da-43ca2180fa57 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:09.863610 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e3862e-e565-429f-83da-43ca2180fa57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:09.882092 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-77e3862e-e565-429f-83da-43ca2180fa57 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:09.883293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-77e3862e-e565-429f-83da-43ca2180fa57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:09.899953 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:09.945153 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1407/2827] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:26:09.952949 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9e1cc-8eaa-43b6-a506-f103fdc961cc None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:09.953265 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9e1cc-8eaa-43b6-a506-f103fdc961cc None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:09.953571 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d9e1cc-8eaa-43b6-a506-f103fdc961cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:09.971911 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-45d9e1cc-8eaa-43b6-a506-f103fdc961cc None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:09.973155 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-45d9e1cc-8eaa-43b6-a506-f103fdc961cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:09.981731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1421/2828] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:26:09.985894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3862e-e565-429f-83da-43ca2180fa57 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:09.995496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e3862e-e565-429f-83da-43ca2180fa57 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:10.000093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b16894-a7f0-4490-99aa-27553dd7cad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:10.000742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b16894-a7f0-4490-99aa-27553dd7cad3 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:10.001056 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b16894-a7f0-4490-99aa-27553dd7cad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:10.001332 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b16894-a7f0-4490-99aa-27553dd7cad3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:10.002057 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b16894-a7f0-4490-99aa-27553dd7cad3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:26:10.002358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b16894-a7f0-4490-99aa-27553dd7cad3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:10.004222 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b16894-a7f0-4490-99aa-27553dd7cad3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:10.015828 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1408/2829] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:26:10.020112 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9e1cc-8eaa-43b6-a506-f103fdc961cc None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:10.029917 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d9e1cc-8eaa-43b6-a506-f103fdc961cc None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:10.034574 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76688c46-91ab-4033-9071-43af45803789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:10.035227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76688c46-91ab-4033-9071-43af45803789 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:10.035512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76688c46-91ab-4033-9071-43af45803789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:10.035788 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76688c46-91ab-4033-9071-43af45803789 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:10.036481 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76688c46-91ab-4033-9071-43af45803789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:10.038409 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76688c46-91ab-4033-9071-43af45803789 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:10.064305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1422/2830] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:26:10 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:26:10.068250 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b16894-a7f0-4490-99aa-27553dd7cad3 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:10.078133 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b16894-a7f0-4490-99aa-27553dd7cad3 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:10.135386 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64088454-dcea-4595-8a26-2b03aa3c45c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:10.136050 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64088454-dcea-4595-8a26-2b03aa3c45c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:10.136331 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64088454-dcea-4595-8a26-2b03aa3c45c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:10.136602 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64088454-dcea-4595-8a26-2b03aa3c45c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:10.137361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64088454-dcea-4595-8a26-2b03aa3c45c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:10.139261 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64088454-dcea-4595-8a26-2b03aa3c45c1 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:10.159102 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1409/2831] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:26:10 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:26:10.163452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76688c46-91ab-4033-9071-43af45803789 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:10.174121 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76688c46-91ab-4033-9071-43af45803789 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:10.178715 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0220d0b-9519-4673-a1d3-40aaaa03491b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:10.179372 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0220d0b-9519-4673-a1d3-40aaaa03491b admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:10.179655 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0220d0b-9519-4673-a1d3-40aaaa03491b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:10.179928 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0220d0b-9519-4673-a1d3-40aaaa03491b admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:10.180689 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0220d0b-9519-4673-a1d3-40aaaa03491b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:10.182302 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0220d0b-9519-4673-a1d3-40aaaa03491b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:10.192845 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1423/2832] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:26:10.196782 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64088454-dcea-4595-8a26-2b03aa3c45c1 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:10.206784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64088454-dcea-4595-8a26-2b03aa3c45c1 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:10.211380 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b3319-efeb-4508-abca-2f2d0a1362e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:10.212034 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b3319-efeb-4508-abca-2f2d0a1362e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:10.212313 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b3319-efeb-4508-abca-2f2d0a1362e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:10.212585 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b3319-efeb-4508-abca-2f2d0a1362e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:10.213366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b3319-efeb-4508-abca-2f2d0a1362e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:10.215418 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9b3319-efeb-4508-abca-2f2d0a1362e0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:10.225197 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1410/2833] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:26:10.229190 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0220d0b-9519-4673-a1d3-40aaaa03491b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:10.242162 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0220d0b-9519-4673-a1d3-40aaaa03491b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:10.246833 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6e223-061b-4f48-9f4d-e8b4d2d44c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:10.247709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6e223-061b-4f48-9f4d-e8b4d2d44c0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:10.247990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6e223-061b-4f48-9f4d-e8b4d2d44c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:10.248262 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f6e223-061b-4f48-9f4d-e8b4d2d44c0b admin admin] PATH_INFO: `/v3/projects/b476fb95f4554a498efd42d47d2c863a/users/a92707737fe84e0e89663a5ae32d834a/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:10.263776 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6e223-061b-4f48-9f4d-e8b4d2d44c0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b476fb95f4554a498efd42d47d2c863a, user_id=a92707737fe84e0e89663a5ae32d834a, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:10.266526 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f6e223-061b-4f48-9f4d-e8b4d2d44c0b admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:10.281981 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1424/2834] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:26:10 2022] PUT /identity/v3/projects/b476fb95f4554a498efd42d47d2c863a/users/a92707737fe84e0e89663a5ae32d834a/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:10.292326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb3885-a59f-4126-86b7-aa5167f5e1ec None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:10.292619 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb3885-a59f-4126-86b7-aa5167f5e1ec None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:10.292897 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb3885-a59f-4126-86b7-aa5167f5e1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:10.319134 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-42bb3885-a59f-4126-86b7-aa5167f5e1ec None None] MFA Rules not processed for user `a92707737fe84e0e89663a5ae32d834a`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:10.320347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-42bb3885-a59f-4126-86b7-aa5167f5e1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:10.385477 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1411/2835] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:26:10.391444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6e223-061b-4f48-9f4d-e8b4d2d44c0b admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:10.412230 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:10.458041 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f6e223-061b-4f48-9f4d-e8b4d2d44c0b admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:10.462573 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9b503-64df-4430-aefc-b81789e37539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:10.463226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9b503-64df-4430-aefc-b81789e37539 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:10.463505 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9b503-64df-4430-aefc-b81789e37539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:10.463775 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9b503-64df-4430-aefc-b81789e37539 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:10.464464 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9b503-64df-4430-aefc-b81789e37539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:10.466943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9b503-64df-4430-aefc-b81789e37539 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:10.491798 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1425/2836] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:26:10 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:26:10.496016 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb3885-a59f-4126-86b7-aa5167f5e1ec None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:10.506999 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb3885-a59f-4126-86b7-aa5167f5e1ec None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:10.519542 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:10.563079 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cb997-3eee-4625-b4c0-1fd011aea6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:10.563742 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cb997-3eee-4625-b4c0-1fd011aea6cd admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:10.564023 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cb997-3eee-4625-b4c0-1fd011aea6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:10.564293 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cb997-3eee-4625-b4c0-1fd011aea6cd admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:10.565008 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cb997-3eee-4625-b4c0-1fd011aea6cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:10.566905 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cb997-3eee-4625-b4c0-1fd011aea6cd admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:10.588532 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1412/2837] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:26:10 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:26:10.593146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9b503-64df-4430-aefc-b81789e37539 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:10.603444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9b503-64df-4430-aefc-b81789e37539 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:10.608009 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256f6a7-1e86-41ff-a8aa-d3f1f9ea56c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:10.608969 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256f6a7-1e86-41ff-a8aa-d3f1f9ea56c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:10.609260 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256f6a7-1e86-41ff-a8aa-d3f1f9ea56c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:10.609530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e256f6a7-1e86-41ff-a8aa-d3f1f9ea56c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:10.610250 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256f6a7-1e86-41ff-a8aa-d3f1f9ea56c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:10.611830 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e256f6a7-1e86-41ff-a8aa-d3f1f9ea56c4 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:10.622954 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1426/2838] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:26:10.627101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cb997-3eee-4625-b4c0-1fd011aea6cd admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:10.637792 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cb997-3eee-4625-b4c0-1fd011aea6cd admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:10.642367 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338fd085-476c-4f63-b7a9-1f5b85c39f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:10.643241 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fd085-476c-4f63-b7a9-1f5b85c39f50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:10.643524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fd085-476c-4f63-b7a9-1f5b85c39f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:10.643802 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338fd085-476c-4f63-b7a9-1f5b85c39f50 admin admin] PATH_INFO: `/v3/projects/5df7fcc78e8d4efd8e3bc54d0d40535f/users/aeb4594c755944c6a3706c3c3c56c4fd/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:10.658921 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338fd085-476c-4f63-b7a9-1f5b85c39f50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df7fcc78e8d4efd8e3bc54d0d40535f, user_id=aeb4594c755944c6a3706c3c3c56c4fd, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:10.661581 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338fd085-476c-4f63-b7a9-1f5b85c39f50 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:10.674174 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1413/2839] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:26:10 2022] PUT /identity/v3/projects/5df7fcc78e8d4efd8e3bc54d0d40535f/users/aeb4594c755944c6a3706c3c3c56c4fd/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:10.678028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256f6a7-1e86-41ff-a8aa-d3f1f9ea56c4 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:10.688450 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e256f6a7-1e86-41ff-a8aa-d3f1f9ea56c4 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:10.742305 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00994f5-49e3-40eb-8f3b-dca23f7c63de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:10.742894 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00994f5-49e3-40eb-8f3b-dca23f7c63de admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:10.743173 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00994f5-49e3-40eb-8f3b-dca23f7c63de admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:10.743445 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00994f5-49e3-40eb-8f3b-dca23f7c63de admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:10.744165 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00994f5-49e3-40eb-8f3b-dca23f7c63de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:10.745787 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00994f5-49e3-40eb-8f3b-dca23f7c63de admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:10.755992 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1427/2840] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:26:10.760230 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338fd085-476c-4f63-b7a9-1f5b85c39f50 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:10.771054 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338fd085-476c-4f63-b7a9-1f5b85c39f50 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:10.775842 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248e385-ce58-4167-afd1-dcd735076f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:10.776763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248e385-ce58-4167-afd1-dcd735076f36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:10.777101 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248e385-ce58-4167-afd1-dcd735076f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:10.777372 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248e385-ce58-4167-afd1-dcd735076f36 admin admin] PATH_INFO: `/v3/projects/5df7fcc78e8d4efd8e3bc54d0d40535f/users/aeb4594c755944c6a3706c3c3c56c4fd/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:10.779421 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248e385-ce58-4167-afd1-dcd735076f36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5df7fcc78e8d4efd8e3bc54d0d40535f, user_id=aeb4594c755944c6a3706c3c3c56c4fd, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:10.782053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1248e385-ce58-4167-afd1-dcd735076f36 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:10.792015 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1414/2841] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:26:10 2022] PUT /identity/v3/projects/5df7fcc78e8d4efd8e3bc54d0d40535f/users/aeb4594c755944c6a3706c3c3c56c4fd/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:10.802948 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c3af6-3644-47c9-b42c-cd1fb73bbcac None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:10.803235 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c3af6-3644-47c9-b42c-cd1fb73bbcac None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:10.803512 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c3af6-3644-47c9-b42c-cd1fb73bbcac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:10.830905 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-ff3c3af6-3644-47c9-b42c-cd1fb73bbcac None None] MFA Rules not processed for user `aeb4594c755944c6a3706c3c3c56c4fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:10.832146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-ff3c3af6-3644-47c9-b42c-cd1fb73bbcac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:10.898484 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1428/2842] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:26:10.909035 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248e385-ce58-4167-afd1-dcd735076f36 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:10.910725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1248e385-ce58-4167-afd1-dcd735076f36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:10.968148 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248e385-ce58-4167-afd1-dcd735076f36 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:10.971927 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc852f2-6d79-49fb-b78d-c8ba022c324d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:10.972555 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc852f2-6d79-49fb-b78d-c8ba022c324d service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:10.972862 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc852f2-6d79-49fb-b78d-c8ba022c324d service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:10.973163 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc852f2-6d79-49fb-b78d-c8ba022c324d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:11.027462 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc852f2-6d79-49fb-b78d-c8ba022c324d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:11.028862 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc852f2-6d79-49fb-b78d-c8ba022c324d service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:11.037564 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1415/2843] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:26:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:26:13.547943 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c3af6-3644-47c9-b42c-cd1fb73bbcac None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:13.565478 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:13.611068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c3af6-3644-47c9-b42c-cd1fb73bbcac None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:13.615583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59284433-5187-4b1b-bd14-93d9a64fa0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:13.616450 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59284433-5187-4b1b-bd14-93d9a64fa0e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:13.616735 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59284433-5187-4b1b-bd14-93d9a64fa0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:13.617037 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59284433-5187-4b1b-bd14-93d9a64fa0e5 admin admin] PATH_INFO: `/v3/users/a92707737fe84e0e89663a5ae32d834a` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:13.618227 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59284433-5187-4b1b-bd14-93d9a64fa0e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a92707737fe84e0e89663a5ae32d834a)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:13.620129 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59284433-5187-4b1b-bd14-93d9a64fa0e5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:13.665581 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-59284433-5187-4b1b-bd14-93d9a64fa0e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a92707737fe84e0e89663a5ae32d834a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:13.666226 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-59284433-5187-4b1b-bd14-93d9a64fa0e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a92707737fe84e0e89663a5ae32d834a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:13.678368 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-59284433-5187-4b1b-bd14-93d9a64fa0e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a92707737fe84e0e89663a5ae32d834a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:13.684999 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-59284433-5187-4b1b-bd14-93d9a64fa0e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a92707737fe84e0e89663a5ae32d834a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:13.691275 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-59284433-5187-4b1b-bd14-93d9a64fa0e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a92707737fe84e0e89663a5ae32d834a'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:13.697285 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1429/2844] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:26:13 2022] DELETE /identity/v3/users/a92707737fe84e0e89663a5ae32d834a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:13.701913 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5927f-ad70-4a52-a6d6-08f950ec8f0e None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:13.702396 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5927f-ad70-4a52-a6d6-08f950ec8f0e None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:13.702516 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5927f-ad70-4a52-a6d6-08f950ec8f0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:13.720328 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-bdb5927f-ad70-4a52-a6d6-08f950ec8f0e None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:13.721932 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-bdb5927f-ad70-4a52-a6d6-08f950ec8f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:13.740894 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:13.782053 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1416/2845] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:26:13.791550 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59284433-5187-4b1b-bd14-93d9a64fa0e5 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:13.793673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-59284433-5187-4b1b-bd14-93d9a64fa0e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:13.850389 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59284433-5187-4b1b-bd14-93d9a64fa0e5 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:13.854043 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc859cc0-833f-4603-98e9-168229d61d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:13.854688 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc859cc0-833f-4603-98e9-168229d61d43 service neutron] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:13.854970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc859cc0-833f-4603-98e9-168229d61d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:13.855242 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc859cc0-833f-4603-98e9-168229d61d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:13.864748 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc859cc0-833f-4603-98e9-168229d61d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:13.866198 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc859cc0-833f-4603-98e9-168229d61d43 service neutron] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:13.875492 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1430/2846] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:26:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:26:14.151068 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5927f-ad70-4a52-a6d6-08f950ec8f0e None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:14.152524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-bdb5927f-ad70-4a52-a6d6-08f950ec8f0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:14.164347 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5927f-ad70-4a52-a6d6-08f950ec8f0e None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:14.169684 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523eb745-4a44-4f6b-84e1-6344c75e0fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:14.170549 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523eb745-4a44-4f6b-84e1-6344c75e0fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:14.170833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523eb745-4a44-4f6b-84e1-6344c75e0fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:14.171113 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523eb745-4a44-4f6b-84e1-6344c75e0fcf admin admin] PATH_INFO: `/v3/projects/b476fb95f4554a498efd42d47d2c863a` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:14.172308 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523eb745-4a44-4f6b-84e1-6344c75e0fcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b476fb95f4554a498efd42d47d2c863a)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:14.174245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523eb745-4a44-4f6b-84e1-6344c75e0fcf admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:14.221155 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-523eb745-4a44-4f6b-84e1-6344c75e0fcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b476fb95f4554a498efd42d47d2c863a', 'request_id': 'req-523eb745-4a44-4f6b-84e1-6344c75e0fcf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:14.227158 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-523eb745-4a44-4f6b-84e1-6344c75e0fcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b476fb95f4554a498efd42d47d2c863a', 'request_id': 'req-523eb745-4a44-4f6b-84e1-6344c75e0fcf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:14.235132 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-523eb745-4a44-4f6b-84e1-6344c75e0fcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b476fb95f4554a498efd42d47d2c863a', 'request_id': 'req-523eb745-4a44-4f6b-84e1-6344c75e0fcf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:14.247243 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-523eb745-4a44-4f6b-84e1-6344c75e0fcf admin admin] The token cache is being invalidate because project b476fb95f4554a498efd42d47d2c863a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:26:14.247538 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-523eb745-4a44-4f6b-84e1-6344c75e0fcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:14.248182 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-523eb745-4a44-4f6b-84e1-6344c75e0fcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:14.249788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1417/2847] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:26:14 2022] DELETE /identity/v3/projects/b476fb95f4554a498efd42d47d2c863a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:14.253755 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc859cc0-833f-4603-98e9-168229d61d43 service neutron] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:14.263539 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc859cc0-833f-4603-98e9-168229d61d43 service neutron] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:14.295756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:14.322778 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198f957-516d-48cd-a3fe-b065a039b731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:14.323558 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198f957-516d-48cd-a3fe-b065a039b731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:14.323839 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198f957-516d-48cd-a3fe-b065a039b731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:14.324107 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198f957-516d-48cd-a3fe-b065a039b731 admin admin] PATH_INFO: `/v3/users/aeb4594c755944c6a3706c3c3c56c4fd` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:14.325456 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198f957-516d-48cd-a3fe-b065a039b731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb4594c755944c6a3706c3c3c56c4fd)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:14.327416 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198f957-516d-48cd-a3fe-b065a039b731 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:14.372873 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e198f957-516d-48cd-a3fe-b065a039b731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb4594c755944c6a3706c3c3c56c4fd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:14.373547 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e198f957-516d-48cd-a3fe-b065a039b731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb4594c755944c6a3706c3c3c56c4fd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:14.385627 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e198f957-516d-48cd-a3fe-b065a039b731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb4594c755944c6a3706c3c3c56c4fd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:14.392342 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e198f957-516d-48cd-a3fe-b065a039b731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb4594c755944c6a3706c3c3c56c4fd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:14.400158 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-e198f957-516d-48cd-a3fe-b065a039b731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb4594c755944c6a3706c3c3c56c4fd'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:14.406612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1431/2848] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:26:14 2022] DELETE /identity/v3/users/aeb4594c755944c6a3706c3c3c56c4fd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:14.672822 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523eb745-4a44-4f6b-84e1-6344c75e0fcf admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:14.674308 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-523eb745-4a44-4f6b-84e1-6344c75e0fcf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:14.687675 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523eb745-4a44-4f6b-84e1-6344c75e0fcf admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:14.745947 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76af857-29f4-4901-bdb9-3111f5dbe091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:14.746814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76af857-29f4-4901-bdb9-3111f5dbe091 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:14.747093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76af857-29f4-4901-bdb9-3111f5dbe091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:14.747366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76af857-29f4-4901-bdb9-3111f5dbe091 admin admin] PATH_INFO: `/v3/projects/5df7fcc78e8d4efd8e3bc54d0d40535f` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:14.748545 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76af857-29f4-4901-bdb9-3111f5dbe091 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5df7fcc78e8d4efd8e3bc54d0d40535f)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:14.750480 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76af857-29f4-4901-bdb9-3111f5dbe091 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:14.796119 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b76af857-29f4-4901-bdb9-3111f5dbe091 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5df7fcc78e8d4efd8e3bc54d0d40535f', 'request_id': 'req-b76af857-29f4-4901-bdb9-3111f5dbe091'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:14.797224 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:14.804089 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b76af857-29f4-4901-bdb9-3111f5dbe091 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5df7fcc78e8d4efd8e3bc54d0d40535f', 'request_id': 'req-b76af857-29f4-4901-bdb9-3111f5dbe091'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:14.811416 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b76af857-29f4-4901-bdb9-3111f5dbe091 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5df7fcc78e8d4efd8e3bc54d0d40535f', 'request_id': 'req-b76af857-29f4-4901-bdb9-3111f5dbe091'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:14.822112 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b76af857-29f4-4901-bdb9-3111f5dbe091 admin admin] The token cache is being invalidate because project 5df7fcc78e8d4efd8e3bc54d0d40535f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80320) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:26:14.822399 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b76af857-29f4-4901-bdb9-3111f5dbe091 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:14.823026 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-b76af857-29f4-4901-bdb9-3111f5dbe091 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:14.825139 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1418/2849] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:26:14 2022] DELETE /identity/v3/projects/5df7fcc78e8d4efd8e3bc54d0d40535f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:14.830026 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26611e-1480-47f4-a826-d19e573d5017 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:14.830514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26611e-1480-47f4-a826-d19e573d5017 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:14.830628 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26611e-1480-47f4-a826-d19e573d5017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:14.849313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-5a26611e-1480-47f4-a826-d19e573d5017 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:14.850444 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-5a26611e-1480-47f4-a826-d19e573d5017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:14.910033 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1432/2850] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:26:14.917754 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9549c-f735-43e6-8e59-9e32e36b9ff4 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:14.918041 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9549c-f735-43e6-8e59-9e32e36b9ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:14.918318 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a9549c-f735-43e6-8e59-9e32e36b9ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:14.936237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-96a9549c-f735-43e6-8e59-9e32e36b9ff4 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:14.937451 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-96a9549c-f735-43e6-8e59-9e32e36b9ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:14.945548 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1419/2851] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:26:14.950127 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26611e-1480-47f4-a826-d19e573d5017 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:14.959641 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26611e-1480-47f4-a826-d19e573d5017 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:14.964146 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6dbd57-ee7f-421f-bee8-c11f0431b5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:14.965450 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dbd57-ee7f-421f-bee8-c11f0431b5dd admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:14.965731 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dbd57-ee7f-421f-bee8-c11f0431b5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:14.965998 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6dbd57-ee7f-421f-bee8-c11f0431b5dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:14.966719 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6dbd57-ee7f-421f-bee8-c11f0431b5dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80321) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:26:14.967018 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6dbd57-ee7f-421f-bee8-c11f0431b5dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:14.968591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6dbd57-ee7f-421f-bee8-c11f0431b5dd admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:14.979837 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1433/2852] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:26:14.983966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9549c-f735-43e6-8e59-9e32e36b9ff4 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:14.993051 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a9549c-f735-43e6-8e59-9e32e36b9ff4 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:14.997588 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38488087-1d53-447b-81f4-b1c6d4078b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:14.998221 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38488087-1d53-447b-81f4-b1c6d4078b6b admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:14.998496 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38488087-1d53-447b-81f4-b1c6d4078b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:14.998766 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38488087-1d53-447b-81f4-b1c6d4078b6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:14.999447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38488087-1d53-447b-81f4-b1c6d4078b6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:15.001346 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38488087-1d53-447b-81f4-b1c6d4078b6b admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:15.027305 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1420/2853] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:26:14 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:26:15.031960 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6dbd57-ee7f-421f-bee8-c11f0431b5dd admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:15.041754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6dbd57-ee7f-421f-bee8-c11f0431b5dd admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:15.084080 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:15.101901 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd891494-4fb2-4172-80d3-0b0eee62efe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:15.102644 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd891494-4fb2-4172-80d3-0b0eee62efe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:15.102931 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd891494-4fb2-4172-80d3-0b0eee62efe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:15.103203 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd891494-4fb2-4172-80d3-0b0eee62efe5 admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:15.103904 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd891494-4fb2-4172-80d3-0b0eee62efe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:15.107087 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd891494-4fb2-4172-80d3-0b0eee62efe5 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:15.128898 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1434/2854] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:26:15 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:26:15.133336 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38488087-1d53-447b-81f4-b1c6d4078b6b admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:15.144506 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38488087-1d53-447b-81f4-b1c6d4078b6b admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:15.149301 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2372741-4204-4c6d-b2c8-834b1a4f06f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:15.150074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2372741-4204-4c6d-b2c8-834b1a4f06f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:15.150458 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2372741-4204-4c6d-b2c8-834b1a4f06f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:15.150832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2372741-4204-4c6d-b2c8-834b1a4f06f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:15.151661 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2372741-4204-4c6d-b2c8-834b1a4f06f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:15.154153 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2372741-4204-4c6d-b2c8-834b1a4f06f5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:15.169345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1421/2855] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:26:15.170068 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd891494-4fb2-4172-80d3-0b0eee62efe5 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:15.180272 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd891494-4fb2-4172-80d3-0b0eee62efe5 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:15.185589 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d0de1-deb4-4e82-b386-fe3448596236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:15.186296 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d0de1-deb4-4e82-b386-fe3448596236 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:15.186530 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d0de1-deb4-4e82-b386-fe3448596236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:15.186805 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d0de1-deb4-4e82-b386-fe3448596236 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:15.187540 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d0de1-deb4-4e82-b386-fe3448596236 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:15.189175 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d0de1-deb4-4e82-b386-fe3448596236 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:15.200265 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1435/2856] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 14 10:26:15.204343 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2372741-4204-4c6d-b2c8-834b1a4f06f5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:15.214763 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2372741-4204-4c6d-b2c8-834b1a4f06f5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:15.220038 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfadd4-8898-4aae-a6df-f30f8f5174f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:15.221054 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfadd4-8898-4aae-a6df-f30f8f5174f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:15.221309 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfadd4-8898-4aae-a6df-f30f8f5174f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:15.221587 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfadd4-8898-4aae-a6df-f30f8f5174f0 admin admin] PATH_INFO: `/v3/projects/dc44bcd296d34369b313d7509cfaca8e/users/8b58fd59f4e0457fbf00742348485022/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:15.237077 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfadd4-8898-4aae-a6df-f30f8f5174f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc44bcd296d34369b313d7509cfaca8e, user_id=8b58fd59f4e0457fbf00742348485022, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:15.239744 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfadd4-8898-4aae-a6df-f30f8f5174f0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:15.254575 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1422/2857] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:26:15 2022] PUT /identity/v3/projects/dc44bcd296d34369b313d7509cfaca8e/users/8b58fd59f4e0457fbf00742348485022/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:15.265750 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ef61d-9d6e-456e-9068-f885713b450b None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:15.266036 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ef61d-9d6e-456e-9068-f885713b450b None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:15.266304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ef61d-9d6e-456e-9068-f885713b450b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:15.293358 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-799ef61d-9d6e-456e-9068-f885713b450b None None] MFA Rules not processed for user `8b58fd59f4e0457fbf00742348485022`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:15.294604 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-799ef61d-9d6e-456e-9068-f885713b450b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:15.366926 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1436/2858] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:26:15.368048 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfadd4-8898-4aae-a6df-f30f8f5174f0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:15.428725 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfadd4-8898-4aae-a6df-f30f8f5174f0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:15.433306 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7fb84-cded-4c2b-9090-ac8d788b037c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:15.433992 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7fb84-cded-4c2b-9090-ac8d788b037c admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:15.434251 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7fb84-cded-4c2b-9090-ac8d788b037c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:15.434523 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d7fb84-cded-4c2b-9090-ac8d788b037c admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:15.435209 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7fb84-cded-4c2b-9090-ac8d788b037c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:15.437095 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d7fb84-cded-4c2b-9090-ac8d788b037c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:15.460759 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1423/2859] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:26:15 2022] POST /identity/v3/projects => generated 377 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:26:15.465344 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ef61d-9d6e-456e-9068-f885713b450b None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:15.475923 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ef61d-9d6e-456e-9068-f885713b450b None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:15.530700 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78b912-ad85-4a0c-b678-79a0968890de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:15.531357 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78b912-ad85-4a0c-b678-79a0968890de admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:15.531649 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78b912-ad85-4a0c-b678-79a0968890de admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:15.531937 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa78b912-ad85-4a0c-b678-79a0968890de admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:15.532612 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78b912-ad85-4a0c-b678-79a0968890de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:15.534545 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa78b912-ad85-4a0c-b678-79a0968890de admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:15.553215 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1437/2860] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:26:15 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:26:15.557300 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7fb84-cded-4c2b-9090-ac8d788b037c admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:15.567352 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d7fb84-cded-4c2b-9090-ac8d788b037c admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:15.571872 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8acd1-4b77-4213-a2df-1a88dfc5c3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:15.572517 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8acd1-4b77-4213-a2df-1a88dfc5c3bc admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:15.572795 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8acd1-4b77-4213-a2df-1a88dfc5c3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:15.573171 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f8acd1-4b77-4213-a2df-1a88dfc5c3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:15.573892 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8acd1-4b77-4213-a2df-1a88dfc5c3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:15.575969 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f8acd1-4b77-4213-a2df-1a88dfc5c3bc admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:15.586464 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1424/2861] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:26:15.591031 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78b912-ad85-4a0c-b678-79a0968890de admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:15.600990 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa78b912-ad85-4a0c-b678-79a0968890de admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:15.605738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991419cd-4dbd-477d-a757-97fd9c2f5303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:15.606383 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991419cd-4dbd-477d-a757-97fd9c2f5303 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:15.606658 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991419cd-4dbd-477d-a757-97fd9c2f5303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:15.606925 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991419cd-4dbd-477d-a757-97fd9c2f5303 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:15.607754 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991419cd-4dbd-477d-a757-97fd9c2f5303 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:15.609367 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991419cd-4dbd-477d-a757-97fd9c2f5303 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:15.619219 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1438/2862] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:26:15.623220 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8acd1-4b77-4213-a2df-1a88dfc5c3bc admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:15.633237 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f8acd1-4b77-4213-a2df-1a88dfc5c3bc admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:15.638282 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16078c62-0656-4616-9e36-fe211f4d38d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:15.639147 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16078c62-0656-4616-9e36-fe211f4d38d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:15.639424 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16078c62-0656-4616-9e36-fe211f4d38d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:15.639694 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16078c62-0656-4616-9e36-fe211f4d38d4 admin admin] PATH_INFO: `/v3/projects/d176dd023e7a4dd8bfbfa90d17b46b07/users/b03d230e83344c298ebdbf0d6bfdfbbe/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:15.654495 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16078c62-0656-4616-9e36-fe211f4d38d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d176dd023e7a4dd8bfbfa90d17b46b07, user_id=b03d230e83344c298ebdbf0d6bfdfbbe, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:15.657199 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16078c62-0656-4616-9e36-fe211f4d38d4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:15.669723 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1425/2863] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:26:15 2022] PUT /identity/v3/projects/d176dd023e7a4dd8bfbfa90d17b46b07/users/b03d230e83344c298ebdbf0d6bfdfbbe/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:15.680756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137af106-a2aa-48f5-b9b4-321b86337c06 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:15.681072 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137af106-a2aa-48f5-b9b4-321b86337c06 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:15.681371 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137af106-a2aa-48f5-b9b4-321b86337c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:15.709487 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-137af106-a2aa-48f5-b9b4-321b86337c06 None None] MFA Rules not processed for user `b03d230e83344c298ebdbf0d6bfdfbbe`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:15.710738 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-137af106-a2aa-48f5-b9b4-321b86337c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:15.729572 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:15.775942 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1439/2864] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:26:15.780986 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16078c62-0656-4616-9e36-fe211f4d38d4 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:15.807299 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:15.843788 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16078c62-0656-4616-9e36-fe211f4d38d4 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:15.848385 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80060f26-0d73-4cce-8fe4-c651daae7ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:15.849067 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80060f26-0d73-4cce-8fe4-c651daae7ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:15.849348 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80060f26-0d73-4cce-8fe4-c651daae7ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:15.849619 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80060f26-0d73-4cce-8fe4-c651daae7ff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:15.850302 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80060f26-0d73-4cce-8fe4-c651daae7ff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:15.852177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80060f26-0d73-4cce-8fe4-c651daae7ff2 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:15.876825 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1426/2865] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:26:15 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:26:15.881079 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137af106-a2aa-48f5-b9b4-321b86337c06 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:15.892351 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137af106-a2aa-48f5-b9b4-321b86337c06 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:15.947092 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff535a-ac4e-4148-a339-6a91f92a9b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:15.947752 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff535a-ac4e-4148-a339-6a91f92a9b2c admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:15.948032 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff535a-ac4e-4148-a339-6a91f92a9b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:15.948304 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff535a-ac4e-4148-a339-6a91f92a9b2c admin admin] PATH_INFO: `/v3/users` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:15.950666 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff535a-ac4e-4148-a339-6a91f92a9b2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:15.951463 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ff535a-ac4e-4148-a339-6a91f92a9b2c admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:15.971868 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1440/2866] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:26:15 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:26:15.975747 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80060f26-0d73-4cce-8fe4-c651daae7ff2 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:15.985695 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80060f26-0d73-4cce-8fe4-c651daae7ff2 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:15.990216 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201526c-b057-4760-bf0b-ae65ad98dc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:15.990863 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201526c-b057-4760-bf0b-ae65ad98dc99 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:15.991142 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201526c-b057-4760-bf0b-ae65ad98dc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:15.991413 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201526c-b057-4760-bf0b-ae65ad98dc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:15.992124 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201526c-b057-4760-bf0b-ae65ad98dc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:15.993783 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201526c-b057-4760-bf0b-ae65ad98dc99 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:16.004554 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1427/2867] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:26:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:26:16.008885 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff535a-ac4e-4148-a339-6a91f92a9b2c admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:16.019361 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff535a-ac4e-4148-a339-6a91f92a9b2c admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:16.023927 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1f4fd-0efc-430b-9c59-05f66de625e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:16.024807 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1f4fd-0efc-430b-9c59-05f66de625e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:16.025137 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1f4fd-0efc-430b-9c59-05f66de625e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:16.025411 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1f4fd-0efc-430b-9c59-05f66de625e8 admin admin] PATH_INFO: `/v3/projects/e4def15948024bce81f575afaa85763b/users/bb9a13147b72428aa1ce2f76be36d7ca/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:16.039888 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1f4fd-0efc-430b-9c59-05f66de625e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4def15948024bce81f575afaa85763b, user_id=bb9a13147b72428aa1ce2f76be36d7ca, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:16.042555 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1f4fd-0efc-430b-9c59-05f66de625e8 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:16.055683 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1441/2868] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:26:16 2022] PUT /identity/v3/projects/e4def15948024bce81f575afaa85763b/users/bb9a13147b72428aa1ce2f76be36d7ca/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:16.060099 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201526c-b057-4760-bf0b-ae65ad98dc99 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:16.069756 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201526c-b057-4760-bf0b-ae65ad98dc99 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:16.126761 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac31c86-85ce-4217-91c9-4d8f709ce7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:16.127487 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac31c86-85ce-4217-91c9-4d8f709ce7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:16.127704 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac31c86-85ce-4217-91c9-4d8f709ce7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:16.127973 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac31c86-85ce-4217-91c9-4d8f709ce7f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:16.128700 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac31c86-85ce-4217-91c9-4d8f709ce7f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:16.130567 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac31c86-85ce-4217-91c9-4d8f709ce7f4 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:16.141183 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1428/2869] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:26:16.145524 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1f4fd-0efc-430b-9c59-05f66de625e8 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:16.156337 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1f4fd-0efc-430b-9c59-05f66de625e8 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:16.160916 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0b48c-6322-4dad-9c2a-c2990f2a8966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:16.161841 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0b48c-6322-4dad-9c2a-c2990f2a8966 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:16.162120 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0b48c-6322-4dad-9c2a-c2990f2a8966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:16.162387 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d0b48c-6322-4dad-9c2a-c2990f2a8966 admin admin] PATH_INFO: `/v3/projects/e4def15948024bce81f575afaa85763b/users/bb9a13147b72428aa1ce2f76be36d7ca/roles/3b5aa52426c743e7904ebec79cd537cb` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:16.164426 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0b48c-6322-4dad-9c2a-c2990f2a8966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4def15948024bce81f575afaa85763b, user_id=bb9a13147b72428aa1ce2f76be36d7ca, role_id=3b5aa52426c743e7904ebec79cd537cb)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:16.167070 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d0b48c-6322-4dad-9c2a-c2990f2a8966 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:16.177082 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1442/2870] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:26:16 2022] PUT /identity/v3/projects/e4def15948024bce81f575afaa85763b/users/bb9a13147b72428aa1ce2f76be36d7ca/roles/3b5aa52426c743e7904ebec79cd537cb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:16.187589 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33952db6-4823-4aa1-aeca-e97594d56d5b None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:16.187879 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33952db6-4823-4aa1-aeca-e97594d56d5b None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:16.188188 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33952db6-4823-4aa1-aeca-e97594d56d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:16.214130 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-33952db6-4823-4aa1-aeca-e97594d56d5b None None] MFA Rules not processed for user `bb9a13147b72428aa1ce2f76be36d7ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:16.215361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-33952db6-4823-4aa1-aeca-e97594d56d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:16.281929 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1429/2871] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:26:16.293542 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0b48c-6322-4dad-9c2a-c2990f2a8966 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:16.295214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-37d0b48c-6322-4dad-9c2a-c2990f2a8966 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:16.351786 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d0b48c-6322-4dad-9c2a-c2990f2a8966 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:16.355968 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9b022-2a4a-44a6-b3c4-60c675dcbf69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:16.356610 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9b022-2a4a-44a6-b3c4-60c675dcbf69 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:16.356890 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9b022-2a4a-44a6-b3c4-60c675dcbf69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:16.357194 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe9b022-2a4a-44a6-b3c4-60c675dcbf69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:16.382638 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:16.413239 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9b022-2a4a-44a6-b3c4-60c675dcbf69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:16.414670 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe9b022-2a4a-44a6-b3c4-60c675dcbf69 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:16.423519 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1443/2872] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:26:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:26:17.890783 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33952db6-4823-4aa1-aeca-e97594d56d5b None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:17.892425 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-33952db6-4823-4aa1-aeca-e97594d56d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:17.904918 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33952db6-4823-4aa1-aeca-e97594d56d5b None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:17.908447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9ee1e-9dae-401f-ac2e-1b8056367bd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:17.909127 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9ee1e-9dae-401f-ac2e-1b8056367bd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:17.909410 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9ee1e-9dae-401f-ac2e-1b8056367bd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:17.909687 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa9ee1e-9dae-401f-ac2e-1b8056367bd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:17.926244 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:17.967090 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9ee1e-9dae-401f-ac2e-1b8056367bd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:17.968441 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa9ee1e-9dae-401f-ac2e-1b8056367bd7 service cinder] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:17.977257 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1430/2873] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:26:17 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:26:18.484799 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9b022-2a4a-44a6-b3c4-60c675dcbf69 service cinder] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:18.493273 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe9b022-2a4a-44a6-b3c4-60c675dcbf69 service cinder] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:18.497400 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8fd4a-ab04-4005-b7c4-4fe0c3f459f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:18.498025 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8fd4a-ab04-4005-b7c4-4fe0c3f459f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:18.498298 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8fd4a-ab04-4005-b7c4-4fe0c3f459f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:18.498567 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8fd4a-ab04-4005-b7c4-4fe0c3f459f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:18.507631 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8fd4a-ab04-4005-b7c4-4fe0c3f459f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:18.509073 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8fd4a-ab04-4005-b7c4-4fe0c3f459f0 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:18.518930 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1444/2874] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:26:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:26:22.330075 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9ee1e-9dae-401f-ac2e-1b8056367bd7 service cinder] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:22.390858 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa9ee1e-9dae-401f-ac2e-1b8056367bd7 service cinder] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:22.395311 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ab4e6-e9cf-468c-b951-9dfbd2505593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:22.396167 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ab4e6-e9cf-468c-b951-9dfbd2505593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:22.396442 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ab4e6-e9cf-468c-b951-9dfbd2505593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:22.396707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04ab4e6-e9cf-468c-b951-9dfbd2505593 admin admin] PATH_INFO: `/v3/users/8b58fd59f4e0457fbf00742348485022` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:22.397929 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ab4e6-e9cf-468c-b951-9dfbd2505593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b58fd59f4e0457fbf00742348485022)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:22.399824 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04ab4e6-e9cf-468c-b951-9dfbd2505593 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:22.448026 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e04ab4e6-e9cf-468c-b951-9dfbd2505593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b58fd59f4e0457fbf00742348485022'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:22.449475 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e04ab4e6-e9cf-468c-b951-9dfbd2505593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b58fd59f4e0457fbf00742348485022'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:22.462542 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e04ab4e6-e9cf-468c-b951-9dfbd2505593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b58fd59f4e0457fbf00742348485022'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:22.470447 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e04ab4e6-e9cf-468c-b951-9dfbd2505593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b58fd59f4e0457fbf00742348485022'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:22.477213 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-e04ab4e6-e9cf-468c-b951-9dfbd2505593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b58fd59f4e0457fbf00742348485022'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:22.483128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1431/2875] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:26:22 2022] DELETE /identity/v3/users/8b58fd59f4e0457fbf00742348485022 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:22.487560 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583607c9-1db4-4fca-b107-b8972c91fdb5 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:22.488021 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583607c9-1db4-4fca-b107-b8972c91fdb5 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:22.488143 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583607c9-1db4-4fca-b107-b8972c91fdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:22.505673 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-583607c9-1db4-4fca-b107-b8972c91fdb5 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:22.506790 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-583607c9-1db4-4fca-b107-b8972c91fdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:22.566255 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1445/2876] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:26:22.576902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ab4e6-e9cf-468c-b951-9dfbd2505593 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:22.578326 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-e04ab4e6-e9cf-468c-b951-9dfbd2505593 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:22.619940 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:22.637267 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04ab4e6-e9cf-468c-b951-9dfbd2505593 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:22.641171 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06682ea-c895-492e-b127-f2d2a8d6c506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:22.641814 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06682ea-c895-492e-b127-f2d2a8d6c506 service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:22.642093 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06682ea-c895-492e-b127-f2d2a8d6c506 service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:22.642366 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06682ea-c895-492e-b127-f2d2a8d6c506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:22.651236 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06682ea-c895-492e-b127-f2d2a8d6c506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:22.652771 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06682ea-c895-492e-b127-f2d2a8d6c506 service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:22.662578 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1432/2877] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:26:22.926573 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583607c9-1db4-4fca-b107-b8972c91fdb5 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:22.928012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-583607c9-1db4-4fca-b107-b8972c91fdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:22.940322 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583607c9-1db4-4fca-b107-b8972c91fdb5 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:22.944949 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401621ca-201d-4c6f-a91d-b163b237b3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:22.945826 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401621ca-201d-4c6f-a91d-b163b237b3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:22.946125 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401621ca-201d-4c6f-a91d-b163b237b3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:22.946394 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401621ca-201d-4c6f-a91d-b163b237b3bb admin admin] PATH_INFO: `/v3/projects/dc44bcd296d34369b313d7509cfaca8e` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:22.947592 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401621ca-201d-4c6f-a91d-b163b237b3bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc44bcd296d34369b313d7509cfaca8e)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:22.950199 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401621ca-201d-4c6f-a91d-b163b237b3bb admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:22.997448 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-401621ca-201d-4c6f-a91d-b163b237b3bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc44bcd296d34369b313d7509cfaca8e', 'request_id': 'req-401621ca-201d-4c6f-a91d-b163b237b3bb'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.003514 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-401621ca-201d-4c6f-a91d-b163b237b3bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc44bcd296d34369b313d7509cfaca8e', 'request_id': 'req-401621ca-201d-4c6f-a91d-b163b237b3bb'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.009414 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-401621ca-201d-4c6f-a91d-b163b237b3bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc44bcd296d34369b313d7509cfaca8e', 'request_id': 'req-401621ca-201d-4c6f-a91d-b163b237b3bb'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.022786 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-401621ca-201d-4c6f-a91d-b163b237b3bb admin admin] The token cache is being invalidate because project dc44bcd296d34369b313d7509cfaca8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:26:23.023070 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-401621ca-201d-4c6f-a91d-b163b237b3bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.023696 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-401621ca-201d-4c6f-a91d-b163b237b3bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.025264 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1446/2878] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:26:22 2022] DELETE /identity/v3/projects/dc44bcd296d34369b313d7509cfaca8e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 14 10:26:23.029404 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06682ea-c895-492e-b127-f2d2a8d6c506 service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:23.039074 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06682ea-c895-492e-b127-f2d2a8d6c506 service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:23.095295 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ca9a1-7859-4224-a49f-8b16818b75f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:23.096179 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ca9a1-7859-4224-a49f-8b16818b75f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:23.096461 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ca9a1-7859-4224-a49f-8b16818b75f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:23.096739 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ca9a1-7859-4224-a49f-8b16818b75f5 admin admin] PATH_INFO: `/v3/users/b03d230e83344c298ebdbf0d6bfdfbbe` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:23.098000 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ca9a1-7859-4224-a49f-8b16818b75f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b03d230e83344c298ebdbf0d6bfdfbbe)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:23.099902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ca9a1-7859-4224-a49f-8b16818b75f5 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:23.145620 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-613ca9a1-7859-4224-a49f-8b16818b75f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b03d230e83344c298ebdbf0d6bfdfbbe'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.146273 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-613ca9a1-7859-4224-a49f-8b16818b75f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b03d230e83344c298ebdbf0d6bfdfbbe'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.159076 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-613ca9a1-7859-4224-a49f-8b16818b75f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b03d230e83344c298ebdbf0d6bfdfbbe'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.165791 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-613ca9a1-7859-4224-a49f-8b16818b75f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b03d230e83344c298ebdbf0d6bfdfbbe'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.177708 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-613ca9a1-7859-4224-a49f-8b16818b75f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b03d230e83344c298ebdbf0d6bfdfbbe'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.183887 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1433/2879] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:26:23 2022] DELETE /identity/v3/users/b03d230e83344c298ebdbf0d6bfdfbbe => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:23.459759 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401621ca-201d-4c6f-a91d-b163b237b3bb admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:23.461352 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-401621ca-201d-4c6f-a91d-b163b237b3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:23.477736 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401621ca-201d-4c6f-a91d-b163b237b3bb admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:23.512276 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:23.537793 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075cb87-cf0e-48e4-bae7-659feb403e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:23.538619 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075cb87-cf0e-48e4-bae7-659feb403e69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:23.538992 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075cb87-cf0e-48e4-bae7-659feb403e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:23.539271 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8075cb87-cf0e-48e4-bae7-659feb403e69 admin admin] PATH_INFO: `/v3/projects/d176dd023e7a4dd8bfbfa90d17b46b07` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:23.540471 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075cb87-cf0e-48e4-bae7-659feb403e69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d176dd023e7a4dd8bfbfa90d17b46b07)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:23.542946 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8075cb87-cf0e-48e4-bae7-659feb403e69 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:23.588503 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8075cb87-cf0e-48e4-bae7-659feb403e69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd176dd023e7a4dd8bfbfa90d17b46b07', 'request_id': 'req-8075cb87-cf0e-48e4-bae7-659feb403e69'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.595036 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8075cb87-cf0e-48e4-bae7-659feb403e69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd176dd023e7a4dd8bfbfa90d17b46b07', 'request_id': 'req-8075cb87-cf0e-48e4-bae7-659feb403e69'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.600555 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8075cb87-cf0e-48e4-bae7-659feb403e69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd176dd023e7a4dd8bfbfa90d17b46b07', 'request_id': 'req-8075cb87-cf0e-48e4-bae7-659feb403e69'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.612941 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8075cb87-cf0e-48e4-bae7-659feb403e69 admin admin] The token cache is being invalidate because project d176dd023e7a4dd8bfbfa90d17b46b07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:26:23.613338 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8075cb87-cf0e-48e4-bae7-659feb403e69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.613988 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-8075cb87-cf0e-48e4-bae7-659feb403e69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.615534 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1447/2880] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:26:23 2022] DELETE /identity/v3/projects/d176dd023e7a4dd8bfbfa90d17b46b07 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:23.619660 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ca9a1-7859-4224-a49f-8b16818b75f5 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:23.621090 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-613ca9a1-7859-4224-a49f-8b16818b75f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:23.633794 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ca9a1-7859-4224-a49f-8b16818b75f5 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:23.689297 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f8808-4dea-41d2-a06b-edea57ad7c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:23.690160 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f8808-4dea-41d2-a06b-edea57ad7c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:23.690440 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f8808-4dea-41d2-a06b-edea57ad7c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:23.690714 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f8808-4dea-41d2-a06b-edea57ad7c1c admin admin] PATH_INFO: `/v3/users/bb9a13147b72428aa1ce2f76be36d7ca` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:23.691884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f8808-4dea-41d2-a06b-edea57ad7c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb9a13147b72428aa1ce2f76be36d7ca)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:23.693833 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f8808-4dea-41d2-a06b-edea57ad7c1c admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:23.695308 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:23.739693 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c81f8808-4dea-41d2-a06b-edea57ad7c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb9a13147b72428aa1ce2f76be36d7ca'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.740345 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c81f8808-4dea-41d2-a06b-edea57ad7c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb9a13147b72428aa1ce2f76be36d7ca'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.752576 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c81f8808-4dea-41d2-a06b-edea57ad7c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb9a13147b72428aa1ce2f76be36d7ca'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.759662 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c81f8808-4dea-41d2-a06b-edea57ad7c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb9a13147b72428aa1ce2f76be36d7ca'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.768177 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-c81f8808-4dea-41d2-a06b-edea57ad7c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb9a13147b72428aa1ce2f76be36d7ca'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:23.774212 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1434/2881] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:26:23 2022] DELETE /identity/v3/users/bb9a13147b72428aa1ce2f76be36d7ca => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:24.043378 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075cb87-cf0e-48e4-bae7-659feb403e69 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:24.044728 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8075cb87-cf0e-48e4-bae7-659feb403e69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:24.057531 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8075cb87-cf0e-48e4-bae7-659feb403e69 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:24.114158 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252fef24-a650-4807-801d-0033a5912591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:24.115028 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fef24-a650-4807-801d-0033a5912591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:24.115312 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fef24-a650-4807-801d-0033a5912591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:24.115591 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fef24-a650-4807-801d-0033a5912591 admin admin] PATH_INFO: `/v3/projects/e4def15948024bce81f575afaa85763b` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:24.116756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fef24-a650-4807-801d-0033a5912591 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4def15948024bce81f575afaa85763b)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:24.118774 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fef24-a650-4807-801d-0033a5912591 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:24.164939 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-252fef24-a650-4807-801d-0033a5912591 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4def15948024bce81f575afaa85763b', 'request_id': 'req-252fef24-a650-4807-801d-0033a5912591'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:24.171124 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-252fef24-a650-4807-801d-0033a5912591 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4def15948024bce81f575afaa85763b', 'request_id': 'req-252fef24-a650-4807-801d-0033a5912591'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:24.176543 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-252fef24-a650-4807-801d-0033a5912591 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4def15948024bce81f575afaa85763b', 'request_id': 'req-252fef24-a650-4807-801d-0033a5912591'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:24.185261 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:24.190709 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-252fef24-a650-4807-801d-0033a5912591 admin admin] The token cache is being invalidate because project e4def15948024bce81f575afaa85763b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:26:24.191214 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-252fef24-a650-4807-801d-0033a5912591 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:24.192295 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-252fef24-a650-4807-801d-0033a5912591 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:24.194085 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1448/2882] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:26:24 2022] DELETE /identity/v3/projects/e4def15948024bce81f575afaa85763b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:24.200601 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8294e-ae12-4152-bd49-7b1d7078053a None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:24.200893 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8294e-ae12-4152-bd49-7b1d7078053a None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:24.201203 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea8294e-ae12-4152-bd49-7b1d7078053a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:24.219834 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-7ea8294e-ae12-4152-bd49-7b1d7078053a None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:24.221115 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-7ea8294e-ae12-4152-bd49-7b1d7078053a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:24.281668 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1435/2883] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:26:24.290424 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed1c1c-d0ba-4be1-b02d-f3fab8d63b11 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:24.290735 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed1c1c-d0ba-4be1-b02d-f3fab8d63b11 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:24.291012 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aed1c1c-d0ba-4be1-b02d-f3fab8d63b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:24.309593 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-7aed1c1c-d0ba-4be1-b02d-f3fab8d63b11 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:24.310723 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-7aed1c1c-d0ba-4be1-b02d-f3fab8d63b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:24.318656 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1449/2884] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 14 10:26:24.322884 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8294e-ae12-4152-bd49-7b1d7078053a None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:24.333128 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea8294e-ae12-4152-bd49-7b1d7078053a None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:24.337664 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e54ca0-99f4-42bb-adce-91b10c4563b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:24.338313 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e54ca0-99f4-42bb-adce-91b10c4563b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:24.338592 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e54ca0-99f4-42bb-adce-91b10c4563b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:24.338866 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e54ca0-99f4-42bb-adce-91b10c4563b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:24.339650 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e54ca0-99f4-42bb-adce-91b10c4563b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80320) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 14 10:26:24.339956 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e54ca0-99f4-42bb-adce-91b10c4563b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:24.341582 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e54ca0-99f4-42bb-adce-91b10c4563b0 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:24.352960 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1436/2885] 10.222.0.61 () {58 vars in 1152 bytes} [Thu Jul 14 10:26:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 14 10:26:24.357265 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed1c1c-d0ba-4be1-b02d-f3fab8d63b11 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:24.366322 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aed1c1c-d0ba-4be1-b02d-f3fab8d63b11 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:24.371398 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4da87-a3c0-4178-9e8a-92b526494205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:24.372036 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4da87-a3c0-4178-9e8a-92b526494205 admin admin] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:24.372313 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4da87-a3c0-4178-9e8a-92b526494205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:24.372583 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef4da87-a3c0-4178-9e8a-92b526494205 admin admin] PATH_INFO: `/v3/projects` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:24.373314 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4da87-a3c0-4178-9e8a-92b526494205 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:24.375186 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef4da87-a3c0-4178-9e8a-92b526494205 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:24.400206 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1450/2886] 10.222.0.61 () {60 vars in 1152 bytes} [Thu Jul 14 10:26:24 2022] POST /identity/v3/projects => generated 389 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 14 10:26:24.404853 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e54ca0-99f4-42bb-adce-91b10c4563b0 admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:24.414832 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e54ca0-99f4-42bb-adce-91b10c4563b0 admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:24.469717 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04127d83-56f0-4682-a81e-85aad77a3d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:24.470368 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04127d83-56f0-4682-a81e-85aad77a3d4e admin admin] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:24.470645 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04127d83-56f0-4682-a81e-85aad77a3d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:24.470915 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04127d83-56f0-4682-a81e-85aad77a3d4e admin admin] PATH_INFO: `/v3/users` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:24.471600 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04127d83-56f0-4682-a81e-85aad77a3d4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:24.473524 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04127d83-56f0-4682-a81e-85aad77a3d4e admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:24.483170 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:24.498784 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1437/2887] 10.222.0.61 () {60 vars in 1143 bytes} [Thu Jul 14 10:26:24 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 14 10:26:24.499970 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4da87-a3c0-4178-9e8a-92b526494205 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:24.510676 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef4da87-a3c0-4178-9e8a-92b526494205 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:24.516050 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587202d9-448a-463c-aaf9-fba7a3fe2b07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:24.516745 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587202d9-448a-463c-aaf9-fba7a3fe2b07 admin admin] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:24.517062 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587202d9-448a-463c-aaf9-fba7a3fe2b07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:24.517343 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587202d9-448a-463c-aaf9-fba7a3fe2b07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:24.518105 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587202d9-448a-463c-aaf9-fba7a3fe2b07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:24.520130 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587202d9-448a-463c-aaf9-fba7a3fe2b07 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:24.531277 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1451/2888] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:26:24.535245 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04127d83-56f0-4682-a81e-85aad77a3d4e admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:24.545663 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04127d83-56f0-4682-a81e-85aad77a3d4e admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:24.550287 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad2ea4-ee72-40c5-9d33-d7ca0376a0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:24.550939 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad2ea4-ee72-40c5-9d33-d7ca0376a0c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:24.551223 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad2ea4-ee72-40c5-9d33-d7ca0376a0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:24.551499 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad2ea4-ee72-40c5-9d33-d7ca0376a0c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:24.552226 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad2ea4-ee72-40c5-9d33-d7ca0376a0c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:24.553848 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad2ea4-ee72-40c5-9d33-d7ca0376a0c3 admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:24.564614 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1438/2889] 10.222.0.61 () {58 vars in 1121 bytes} [Thu Jul 14 10:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 14 10:26:24.568554 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587202d9-448a-463c-aaf9-fba7a3fe2b07 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:24.578359 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587202d9-448a-463c-aaf9-fba7a3fe2b07 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:24.583324 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7ed11-6a13-4b84-9e52-8faf0b066926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:24.584196 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7ed11-6a13-4b84-9e52-8faf0b066926 admin admin] REQUEST_METHOD: `PUT` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:24.584473 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7ed11-6a13-4b84-9e52-8faf0b066926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:24.584741 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb7ed11-6a13-4b84-9e52-8faf0b066926 admin admin] PATH_INFO: `/v3/projects/ad220945ce0247c7a74657f7ff9f3aac/users/4ee443e39dc04e78862b2d5096dd9fcf/roles/3d9688bbbc15422cbfc59ded082fbe41` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:24.599163 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7ed11-6a13-4b84-9e52-8faf0b066926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad220945ce0247c7a74657f7ff9f3aac, user_id=4ee443e39dc04e78862b2d5096dd9fcf, role_id=3d9688bbbc15422cbfc59ded082fbe41)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:24.601857 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb7ed11-6a13-4b84-9e52-8faf0b066926 admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:24.615433 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1452/2890] 10.222.0.61 () {60 vars in 1482 bytes} [Thu Jul 14 10:26:24 2022] PUT /identity/v3/projects/ad220945ce0247c7a74657f7ff9f3aac/users/4ee443e39dc04e78862b2d5096dd9fcf/roles/3d9688bbbc15422cbfc59ded082fbe41 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:24.626452 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3242d-b381-44c8-887e-cbb0e1d8a7e8 None None] REQUEST_METHOD: `POST` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:24.627403 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3242d-b381-44c8-887e-cbb0e1d8a7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:24.627753 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3242d-b381-44c8-887e-cbb0e1d8a7e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:24.654726 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.auth.core [None req-bde3242d-b381-44c8-887e-cbb0e1d8a7e8 None None] MFA Rules not processed for user `4ee443e39dc04e78862b2d5096dd9fcf`. Rule list: `[]` (Enabled: `True`). {{(pid=80320) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:24.655934 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-bde3242d-b381-44c8-887e-cbb0e1d8a7e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:24.718361 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1439/2891] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:26:24.729075 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7ed11-6a13-4b84-9e52-8faf0b066926 admin admin] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:24.731382 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-8bb7ed11-6a13-4b84-9e52-8faf0b066926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:24.787904 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb7ed11-6a13-4b84-9e52-8faf0b066926 admin admin] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:24.791516 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38702fee-9bd7-4edd-b8c4-a06c7107c651 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '52e3fbd0400a49a69932ce426218f4a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:24.792152 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38702fee-9bd7-4edd-b8c4-a06c7107c651 service cinder] REQUEST_METHOD: `GET` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:24.792986 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38702fee-9bd7-4edd-b8c4-a06c7107c651 service cinder] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:24.793263 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38702fee-9bd7-4edd-b8c4-a06c7107c651 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:24.801419 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38702fee-9bd7-4edd-b8c4-a06c7107c651 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:24.802833 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38702fee-9bd7-4edd-b8c4-a06c7107c651 service cinder] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:24.811992 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1453/2892] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:26:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:26:34.715489 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=39 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:34.807452 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:40.923598 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3242d-b381-44c8-887e-cbb0e1d8a7e8 None None] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:40.983696 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3242d-b381-44c8-887e-cbb0e1d8a7e8 None None] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:40.989274 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9933cd4-a7e2-4898-8aea-673a8032823d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:40.990211 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9933cd4-a7e2-4898-8aea-673a8032823d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:40.990526 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9933cd4-a7e2-4898-8aea-673a8032823d admin admin] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:40.990802 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9933cd4-a7e2-4898-8aea-673a8032823d admin admin] PATH_INFO: `/v3/users/4ee443e39dc04e78862b2d5096dd9fcf` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:40.992027 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9933cd4-a7e2-4898-8aea-673a8032823d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ee443e39dc04e78862b2d5096dd9fcf)` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:40.994013 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9933cd4-a7e2-4898-8aea-673a8032823d admin admin] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:41.040329 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9933cd4-a7e2-4898-8aea-673a8032823d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ee443e39dc04e78862b2d5096dd9fcf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:41.041006 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9933cd4-a7e2-4898-8aea-673a8032823d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ee443e39dc04e78862b2d5096dd9fcf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:41.053311 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9933cd4-a7e2-4898-8aea-673a8032823d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ee443e39dc04e78862b2d5096dd9fcf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:41.059966 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9933cd4-a7e2-4898-8aea-673a8032823d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ee443e39dc04e78862b2d5096dd9fcf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:41.067862 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.notifications [None req-a9933cd4-a7e2-4898-8aea-673a8032823d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ee443e39dc04e78862b2d5096dd9fcf'} {{(pid=80320) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:41.074902 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1440/2893] 10.222.0.61 () {58 vars in 1223 bytes} [Thu Jul 14 10:26:40 2022] DELETE /identity/v3/users/4ee443e39dc04e78862b2d5096dd9fcf => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:41.079457 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0045c-27b3-416f-aa69-c7cda5775e76 None None] REQUEST_METHOD: `POST` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:41.079951 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0045c-27b3-416f-aa69-c7cda5775e76 None None] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:41.080054 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf0045c-27b3-416f-aa69-c7cda5775e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:41.097852 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.auth.core [None req-bdf0045c-27b3-416f-aa69-c7cda5775e76 None None] MFA Rules not processed for user `e054eebdda2a494f804b604ef19ea604`. Rule list: `[]` (Enabled: `True`). {{(pid=80321) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 14 10:26:41.099033 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bdf0045c-27b3-416f-aa69-c7cda5775e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:41.158643 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1454/2894] 10.222.0.61 () {58 vars in 957 bytes} [Thu Jul 14 10:26:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 14 10:26:41.168469 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9933cd4-a7e2-4898-8aea-673a8032823d admin admin] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:41.169904 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-a9933cd4-a7e2-4898-8aea-673a8032823d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:41.225611 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9933cd4-a7e2-4898-8aea-673a8032823d admin admin] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:41.229319 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96ca8d-fa24-49d5-be87-b9d896566dea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cd6d19c4406b4b228065b98a9a074b3a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:41.229954 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96ca8d-fa24-49d5-be87-b9d896566dea service neutron] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:41.230235 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96ca8d-fa24-49d5-be87-b9d896566dea service neutron] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:41.230511 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a96ca8d-fa24-49d5-be87-b9d896566dea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:41.239785 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96ca8d-fa24-49d5-be87-b9d896566dea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:41.241249 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a96ca8d-fa24-49d5-be87-b9d896566dea service neutron] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:41.250942 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1441/2895] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:26:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:26:41.519125 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0045c-27b3-416f-aa69-c7cda5775e76 None None] Authenticating user token {{(pid=80321) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:26:41.520600 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.fernet_utils [None req-bdf0045c-27b3-416f-aa69-c7cda5775e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80321) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:26:41.537880 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf0045c-27b3-416f-aa69-c7cda5775e76 None None] Validating token access rules against request {{(pid=80321) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:26:41.542870 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e479cc-46fd-4b9e-bb97-64676ca576fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e054eebdda2a494f804b604ef19ea604', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4677fbdc96dd49fca8554ea63b0d5a7c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80321) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:26:41.543784 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e479cc-46fd-4b9e-bb97-64676ca576fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:26:41.544066 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e479cc-46fd-4b9e-bb97-64676ca576fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:26:41.544364 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e479cc-46fd-4b9e-bb97-64676ca576fc admin admin] PATH_INFO: `/v3/projects/ad220945ce0247c7a74657f7ff9f3aac` {{(pid=80321) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:26:41.545528 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e479cc-46fd-4b9e-bb97-64676ca576fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad220945ce0247c7a74657f7ff9f3aac)` {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:26:41.547422 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e479cc-46fd-4b9e-bb97-64676ca576fc admin admin] RBAC: Authorization granted {{(pid=80321) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:26:41.594800 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-38e479cc-46fd-4b9e-bb97-64676ca576fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad220945ce0247c7a74657f7ff9f3aac', 'request_id': 'req-38e479cc-46fd-4b9e-bb97-64676ca576fc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:41.600762 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-38e479cc-46fd-4b9e-bb97-64676ca576fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad220945ce0247c7a74657f7ff9f3aac', 'request_id': 'req-38e479cc-46fd-4b9e-bb97-64676ca576fc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:41.606667 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-38e479cc-46fd-4b9e-bb97-64676ca576fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad220945ce0247c7a74657f7ff9f3aac', 'request_id': 'req-38e479cc-46fd-4b9e-bb97-64676ca576fc'} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:41.618756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-38e479cc-46fd-4b9e-bb97-64676ca576fc admin admin] The token cache is being invalidate because project ad220945ce0247c7a74657f7ff9f3aac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80321) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 14 10:26:41.619154 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-38e479cc-46fd-4b9e-bb97-64676ca576fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:41.620341 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG keystone.notifications [None req-38e479cc-46fd-4b9e-bb97-64676ca576fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80321) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 14 10:26:41.621989 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: [pid: 80321|app: 0|req: 1455/2896] 10.222.0.61 () {58 vars in 1232 bytes} [Thu Jul 14 10:26:41 2022] DELETE /identity/v3/projects/ad220945ce0247c7a74657f7ff9f3aac => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 14 10:26:51.246707 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:26:51.615756 dv-cinder-849800-1-iscsi devstack@keystone.service[80321]: DEBUG dbcounter [-] [80321] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80321) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 14 10:27:02.273556 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96ca8d-fa24-49d5-be87-b9d896566dea service neutron] Authenticating user token {{(pid=80320) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 14 10:27:02.275568 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-0a96ca8d-fa24-49d5-be87-b9d896566dea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:27:02.342752 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a96ca8d-fa24-49d5-be87-b9d896566dea service neutron] Validating token access rules against request {{(pid=80320) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 14 10:27:02.400455 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f122c27-10d7-427c-9728-32f10a28c915 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ff4c42a4f4f549378e5bc1cd92f801c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '57662256880f43018039715a755c087c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80320) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 14 10:27:02.401131 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f122c27-10d7-427c-9728-32f10a28c915 service placement] REQUEST_METHOD: `GET` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 14 10:27:02.401412 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f122c27-10d7-427c-9728-32f10a28c915 service placement] SCRIPT_NAME: `/identity` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 14 10:27:02.401681 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f122c27-10d7-427c-9728-32f10a28c915 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80320) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 14 10:27:02.403430 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.fernet_utils [None req-1f122c27-10d7-427c-9728-32f10a28c915 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80320) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 14 10:27:02.414358 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f122c27-10d7-427c-9728-32f10a28c915 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 14 10:27:02.415950 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f122c27-10d7-427c-9728-32f10a28c915 service placement] RBAC: Authorization granted {{(pid=80320) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 14 10:27:02.425689 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: [pid: 80320|app: 0|req: 1442/2897] 10.222.0.61 () {60 vars in 1365 bytes} [Thu Jul 14 10:27:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 14 10:27:12.421497 dv-cinder-849800-1-iscsi devstack@keystone.service[80320]: DEBUG dbcounter [-] [80320] Writing DB stats keystone:SELECT=58 {{(pid=80320) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}